-- Logs begin at Tue 2022-06-07 07:46:26 UTC, end at Tue 2022-06-07 09:28:53 UTC. -- Jun 07 08:20:17.410506 dv-cinder-733062-30-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 07 08:20:17.422887 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 07 08:20:17.426073 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 7 08:20:17 2022] *** Jun 07 08:20:17.426301 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 07 08:20:17.426463 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 07 08:20:17.426639 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: nodename: dv-cinder-733062-30-iscsi Jun 07 08:20:17.426790 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: machine: x86_64 Jun 07 08:20:17.426960 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: clock source: unix Jun 07 08:20:17.427118 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: pcre jit disabled Jun 07 08:20:17.427279 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: detected number of CPU cores: 4 Jun 07 08:20:17.427428 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: current working directory: / Jun 07 08:20:17.427577 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: detected binary path: /usr/bin/uwsgi-core Jun 07 08:20:17.427731 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: your processes number limit is 47680 Jun 07 08:20:17.427881 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: your memory page size is 4096 bytes Jun 07 08:20:17.428058 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: detected max file descriptor number: 2048 Jun 07 08:20:17.428232 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: lock engine: pthread robust mutexes Jun 07 08:20:17.428621 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: thunder lock: enabled Jun 07 08:20:17.429035 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 07 08:20:17.429358 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 07 08:20:17.483294 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: Python main interpreter initialized at 0x563f7a00d9c0 Jun 07 08:20:17.483294 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: python threads support enabled Jun 07 08:20:17.483294 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: your server socket listen backlog is limited to 100 connections Jun 07 08:20:17.483294 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: your mercy for graceful operations on workers is 90 seconds Jun 07 08:20:17.483678 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: mapped 403077 bytes (393 KB) for 2 cores Jun 07 08:20:17.483756 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: *** Operational MODE: preforking *** Jun 07 08:20:17.484024 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: *** uWSGI is running in multiple interpreter mode *** Jun 07 08:20:17.484024 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: spawned uWSGI master process (pid: 80218) Jun 07 08:20:17.484254 dv-cinder-733062-30-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 07 08:20:17.488174 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: spawned uWSGI worker 1 (pid: 80228, cores: 1) Jun 07 08:20:17.488174 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: spawned uWSGI worker 2 (pid: 80229, cores: 1) Jun 07 08:20:17.488174 dv-cinder-733062-30-iscsi devstack@keystone.service[80218]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 07 08:20:23.560703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 08:20:23.560703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: from cryptography.utils import int_from_bytes Jun 07 08:20:23.561681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 08:20:23.561681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: from cryptography.utils import int_from_bytes Jun 07 08:20:23.570582 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 08:20:23.570582 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: from cryptography.utils import int_from_bytes Jun 07 08:20:23.571205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 08:20:23.571205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: from cryptography.utils import int_from_bytes Jun 07 08:20:23.615799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 08:20:23.616033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 08:20:23.616176 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] command line args: [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 08:20:23.616416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 08:20:23.616600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ================================================================================ {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 08:20:23.616854 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] admin_token = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.617109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] config_dir = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.617388 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.617542 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] config_source = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.617641 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] debug = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.617832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.619329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.619329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.619329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.619329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.619329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.619329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_config_append = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.619687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.619687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_dir = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.619687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_file = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.619938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.620182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.620379 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.620626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.620789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.621033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.621233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.621540 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.621720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.621943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.622153 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.622908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.623028 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 08:20:23.623247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.623361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 08:20:23.623651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.623844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] command line args: [] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 08:20:23.624195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.624365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 08:20:23.624365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ================================================================================ {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 08:20:23.624588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.624588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] publish_errors = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.625306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] admin_token = **** {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.625562 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.625749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] config_dir = [] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.625836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.625947 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.626025 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.626130 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] config_source = [] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.626130 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] debug = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.626300 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.626381 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.626491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.626491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] standard_threads = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.626753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.626753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.626953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.627041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.627041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.627176 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.627254 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] list_limit = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.627339 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.627499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] log_config_append = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.627588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] use_journal = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.627679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.627762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] use_json = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.627836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] log_dir = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.627919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] use_stderr = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.627983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] log_file = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.628057 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] use_syslog = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.628130 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.628310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.628399 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.628504 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.628568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.628633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.628710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.628773 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.628883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.629012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.629145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.629288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.629502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.629628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.629919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.630015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.630086 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.630178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.630273 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.630273 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.630404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.630486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.630553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.630630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.630717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.630854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.630967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.630967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.631112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.631284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.631284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.631427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.631510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.631584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] publish_errors = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.631654 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.631714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.631778 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.631945 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.631945 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.632083 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.632150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.632317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.632412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.632624 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.632706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] standard_threads = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.632785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.632863 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.632938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.633178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.633282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.633375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.633375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] use_journal = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.633525 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.633615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] use_json = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.633709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.633783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] use_stderr = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.633860 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.633932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] use_syslog = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.634049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.634235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:20:23.634356 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.634438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.634515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.634592 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.634661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.634661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.634799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.634864 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.634921 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.634978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.635035 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.635193 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.635295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.635370 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.635495 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.635670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.635787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.635874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.635950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.636048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.636152 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.636253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.636342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.636464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.636620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.636724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.636788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.636851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.637023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.637134 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.637330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.637460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.637532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.637664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.637727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.637840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.637978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.638102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.638169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.638301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.638405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.638484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.638713 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.638844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.638928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.639008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.639118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.639242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.639351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.639501 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.639581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.639734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.639821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.639969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.640065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.640225 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.640330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.640420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.641012 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.641249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.641351 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.641677 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.641771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.641856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.642023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.642130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.642277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.642363 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.642527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.642693 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.642781 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.642849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.643045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.643125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.643216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.643285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.643505 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.643623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.643695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.643858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.644056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.644258 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.644450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.644617 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.644724 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.644826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.644991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.645155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.645266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.645516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.645721 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.645939 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.646146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.646628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.646910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.647131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.647347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.647631 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.647894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.648115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.648332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.648714 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.648897 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.649222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.649414 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.649546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.649808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.650023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.650162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.650502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.650637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.650758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.650888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.651107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.651240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.651455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.651587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.651722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.652358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.652460 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.652540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.652625 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.652704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.652777 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.652850 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.652917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.653149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.653244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.653481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.653569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.653631 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.653704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.653775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.653857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.653937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.654011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.654082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.654144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.654214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.654294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.654426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.654512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.654661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.654784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.654899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.654984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.655116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.655201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.655276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.655410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.655489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.655560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.external = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.655631 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.655701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.655763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.655823 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.655890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.656017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.656089 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.password = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.656089 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] auth.token = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.656893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.656955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.657023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.657115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.657181 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.657241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.657367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.657435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.657494 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.657579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.657748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.657822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.657882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.657941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658072 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658134 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658193 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658504 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658656 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658727 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.658976 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659814 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.659990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.660171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.660242 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.660303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] auth.external = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.660364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.660429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.660490 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.660557 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.660621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.660621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.660727 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.660846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] auth.password = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.660933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.661065 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.661218 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] auth.token = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.661400 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.661486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.661572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.661657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.661829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.661900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.662066 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.662144 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.662312 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.662395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.662548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.662629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.662785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.662955 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.663064 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.663136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] credential.caching = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.663262 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.663339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.663554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.663636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.663746 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.663898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.663965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.664026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.664200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.664337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.664454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.664603 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.664689 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.664752 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.665027 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.665147 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.665232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.665318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.665390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.665550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.665633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.665845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.665915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.666094 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.666166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.666376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.666445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.666661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.666742 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.666894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.667004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.667114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.667245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] federation.caching = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.667349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.667418 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.667578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.667657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.667811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.667883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.668070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.668149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.668272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.668372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.668513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.668642 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.668749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.668856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.669008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.669152 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.669224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.669364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.669512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.669592 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.669715 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.669837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.669948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.670098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.caching = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.670192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.670362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.670440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.670653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.670723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.670889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.670959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.671188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.671283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.671349 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.671595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.671869 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.672185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.672553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.673020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.673306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.673398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.673583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.673731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.673811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.673887 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.674047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.674250 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.674334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.674397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.674517 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.674650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.674744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.674940 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.675030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.675186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.675283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.675421 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.675512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.675680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.675816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.675907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.676072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.676154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.676357 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.676515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.676597 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.676671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.676855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.676947 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.677049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.677290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.677376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.677440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.677559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.677645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.677795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.677867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.678029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.678230 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.678321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.678397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.678536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.678628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.678747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.678838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.679027 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.679121 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.679239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.679329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.679461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.679543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.679706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.679828 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.679942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.680071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.680224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.680314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.680459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.680604 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.680694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.680842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.680923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.681110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.681215 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.681381 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.681462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.681668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.681747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.681896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.681984 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.682139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.682220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.682374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.682463 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.682607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.682706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.682864 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.682987 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.683173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.683254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.683451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.683550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.683705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.683825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.683931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.684013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.684172 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.684264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.684424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.684512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.684672 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.684752 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.684907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.684997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.685235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.685329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.685441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.685522 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.685675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.685769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.685919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.686045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.686154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.686304 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.686393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.686557 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.686637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] role.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.686847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] role.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.687077 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.687165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.687831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.687986 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.688132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.688368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.688461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.688615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.688875 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.688955 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.689324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.689426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.689577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.689871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.689956 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.690054 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.690118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.690179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.690287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.690457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.690550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.690711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.690847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.690948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.691030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.691262 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.691412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.691493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.691652 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.691727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.691880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.691972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.692100 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.692178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.692387 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.692577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.692577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.692820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.692820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.692947 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.693110 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.693199 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.693336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.693429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.693592 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.693748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.693822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.693883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.694043 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.694139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.694273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.694354 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.694502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.694592 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.694751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.694842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.695007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.695088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.695256 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.695347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.695492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.695698 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] resource.caching = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.695785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.695860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.695932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.696036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.696185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.696311 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.696394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.696564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.696644 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.696849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.696978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.697166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.697270 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.697336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.697396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.697562 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.697659 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.697803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.697883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.698045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.698134 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] role.caching = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.698289 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.698453 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] role.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.698534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.698607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.698742 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.698872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.698974 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.699126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.699208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.699422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.699514 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.699628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.699709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.699880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.700025 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.700128 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.700209 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.700401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.700493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.connection = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.700620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.700755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.700843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.700921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.701108 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.701187 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.701394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.701490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.701568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.701689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.701802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.701874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.702036 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.702268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.702349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.702510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.702591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.702743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.703072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.703319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.703411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.703585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.703665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.704018 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.704098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.704340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.704505 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.704702 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.704842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.704933 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.705046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.705163 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.705326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.705407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.705564 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.705666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.705737 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80228) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 08:20:23.705963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.706163 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.706462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.706716 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.706954 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.707192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.707496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.707660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] token.caching = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.707897 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.708150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.708387 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.708644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.708893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.709156 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.709405 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.709650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.709898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.710134 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.710373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.710610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.710856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.711152 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.711388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.711677 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.712053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.712325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.connection = **** {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.712570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.712865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.713126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.713413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.713652 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.713936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.714177 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.714455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.714781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.715106 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.715292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.715529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.715803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.716056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.716303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.716580 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.716856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.717117 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:20:23.717294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80229) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 08:20:23.736881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.737223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.737596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.746912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.756684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.756993 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.757326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.770215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.777906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.778126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.799007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.799295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.836175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.847316 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.887632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.887861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.889773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.890005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.961616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.961820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.962048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.962263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.962479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.962669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.962887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.962975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.963129 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.963335 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.963599 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.963905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.988016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.988269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.988417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.988557 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.988812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.988956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.989134 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.989134 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.989485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.989565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.989657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.989748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.989882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:23.989882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.001883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.002635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.015027 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.015283 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.015563 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.015786 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.015924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.016115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.016215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.016440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.016544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.016662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.016857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80228) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.016965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.017189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.017403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80229) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:20:24.035587 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.035841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 08:20:24.036116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.036316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 08:20:24.036586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.036856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.037133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.037266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.037396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 08:20:24.037522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.037758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.037893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 08:20:24.038174 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.038440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.038706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.038999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.049105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.049384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.049654 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.049936 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.051250 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.051566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.051833 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.052110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.058240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.060676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.061538 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.061870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.062148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.062455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.062980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.063402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.063800 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.064013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.064251 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.064383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.064532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.064623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.064800 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.064906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.065104 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.065393 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.065654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.065924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.066268 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.066542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.066811 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.067078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.092675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.095035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.095310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.096086 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.097787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.098065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.100629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.100911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.101369 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.103881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.104197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.104520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.108585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.108865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.112649 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.112999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.113831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.114118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.114399 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.114674 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.114953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.115220 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.115480 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.115740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.117965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.118291 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.118576 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.118898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.119182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.119496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.119764 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.120038 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.134843 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.135124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.139708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.140334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.141202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.144237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.144529 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.146520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.150661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.151004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.151390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.155100 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.157327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.159998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.160381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.160665 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.160945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.161255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.161541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.161631 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.165763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.166175 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.166468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.166749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.167050 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.167352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.181712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.182048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.182320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.182575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.188529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.188834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.189202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.189536 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.191425 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.193286 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.193620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.197040 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.197345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.197624 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.197916 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.198197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.198431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.201750 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.202027 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.206425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.206719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.206999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.207368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.207716 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.212920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.213237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.213524 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.213804 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.214086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.214463 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.214753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.223170 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.223466 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.223822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.224146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.224502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.224855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.225198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.234013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.237765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.241234 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.242995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.244623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.244897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.245108 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.245354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.248610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.252358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.253204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.254207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.254861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.255922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.256237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.256527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.258190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.258474 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.258876 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.259162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.264605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.267071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.269225 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.269521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.269799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.270574 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.270950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.271287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.271580 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.277741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80228) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.278023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.278306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.278579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.278848 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.279242 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.279517 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.279781 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.280048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.280316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.280619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.280943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.281896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.282302 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.282506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80228) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.282668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.283030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.291452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80229) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:20:24.291706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.292112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.292372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.292605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.292851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.293196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.293472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.293741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.294069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.294340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.294641 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.294885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.295334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80229) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:20:24.321615 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x563f7a00d9c0 pid: 80228 (default app) Jun 07 08:20:24.325182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438020e0-9ed4-40b5-86dc-be1c7c682e32 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:24.325523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438020e0-9ed4-40b5-86dc-be1c7c682e32 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:24.325752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438020e0-9ed4-40b5-86dc-be1c7c682e32 None None] PATH_INFO: `/v3` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:24.331407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 791 bytes} [Tue Jun 7 08:20:24 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:24.336489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x563f7a00d9c0 pid: 80229 (default app) Jun 07 08:20:32.856052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2edf3-1041-4d04-a082-afdbd9f438e5 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:32.856618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2edf3-1041-4d04-a082-afdbd9f438e5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:32.856618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2edf3-1041-4d04-a082-afdbd9f438e5 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:32.857951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 2/2] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:32.865708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4ce11-56e0-444c-a6e8-0314ed30d355 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:32.866239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4ce11-56e0-444c-a6e8-0314ed30d355 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:32.866332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4ce11-56e0-444c-a6e8-0314ed30d355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:32.934335 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: INFO dbcounter [None req-dbb4ce11-56e0-444c-a6e8-0314ed30d355 None None] Registered counter for database keystone Jun 07 08:20:32.956894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG oslo_db.sqlalchemy.engines [None req-dbb4ce11-56e0-444c-a6e8-0314ed30d355 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80229) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 08:20:32.963277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writer thread running {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 08:20:33.249507 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG passlib.handlers.bcrypt [None req-dbb4ce11-56e0-444c-a6e8-0314ed30d355 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80229) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 08:20:33.249944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG passlib.handlers.bcrypt [None req-dbb4ce11-56e0-444c-a6e8-0314ed30d355 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80229) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 08:20:33.306716 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-dbb4ce11-56e0-444c-a6e8-0314ed30d355 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:33.308345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-dbb4ce11-56e0-444c-a6e8-0314ed30d355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:33.458435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1/3] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 596 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 07 08:20:33.654540 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2edf3-1041-4d04-a082-afdbd9f438e5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:33.727020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: INFO dbcounter [None req-bdb2edf3-1041-4d04-a082-afdbd9f438e5 None None] Registered counter for database keystone Jun 07 08:20:33.756473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG oslo_db.sqlalchemy.engines [None req-bdb2edf3-1041-4d04-a082-afdbd9f438e5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80228) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 08:20:33.763076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writer thread running {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 08:20:33.892667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2edf3-1041-4d04-a082-afdbd9f438e5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:33.897258 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd9082-88c9-4f7d-9f8f-ad777bde5d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:33.897947 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd9082-88c9-4f7d-9f8f-ad777bde5d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:33.898229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd9082-88c9-4f7d-9f8f-ad777bde5d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:33.898502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd9082-88c9-4f7d-9f8f-ad777bde5d12 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:33.907803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd9082-88c9-4f7d-9f8f-ad777bde5d12 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:33.960023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: WARNING keystone.common.rbac_enforcer.enforcer [None req-02bd9082-88c9-4f7d-9f8f-ad777bde5d12 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 08:20:33.960360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd9082-88c9-4f7d-9f8f-ad777bde5d12 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:33.970838 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 3/4] 10.222.0.39 () {58 vars in 1324 bytes} [Tue Jun 7 08:20:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:20:33.976891 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4ce11-56e0-444c-a6e8-0314ed30d355 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:33.992037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4ce11-56e0-444c-a6e8-0314ed30d355 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:33.996575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31fe36-f9f2-48f4-84c1-607c3ac1bc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:33.997741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31fe36-f9f2-48f4-84c1-607c3ac1bc23 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:33.998028 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31fe36-f9f2-48f4-84c1-607c3ac1bc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:33.998307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31fe36-f9f2-48f4-84c1-607c3ac1bc23 admin admin] PATH_INFO: `/v3/projects/ddc110e64f52436a913830733ab91055` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:34.000554 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31fe36-f9f2-48f4-84c1-607c3ac1bc23 admin admin] RBAC: Authorizing `identity:get_project(project_id=ddc110e64f52436a913830733ab91055)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:34.055130 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dc31fe36-f9f2-48f4-84c1-607c3ac1bc23 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 08:20:34.055477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31fe36-f9f2-48f4-84c1-607c3ac1bc23 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:34.060234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 2/5] 10.222.0.39 () {56 vars in 1207 bytes} [Tue Jun 7 08:20:33 2022] GET /identity/v3/projects/ddc110e64f52436a913830733ab91055 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:36.321182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cac53-045e-4d30-8145-97978b368d6c None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.321682 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cac53-045e-4d30-8145-97978b368d6c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.321682 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cac53-045e-4d30-8145-97978b368d6c None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.325851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 4/6] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:36.332099 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69721fdf-d4d5-4263-beea-67e09969dd81 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.332550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69721fdf-d4d5-4263-beea-67e09969dd81 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.332642 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69721fdf-d4d5-4263-beea-67e09969dd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.353147 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-69721fdf-d4d5-4263-beea-67e09969dd81 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:36.354685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-69721fdf-d4d5-4263-beea-67e09969dd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:36.362667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 3/7] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 07 08:20:36.385541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cac53-045e-4d30-8145-97978b368d6c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:36.398380 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cac53-045e-4d30-8145-97978b368d6c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:36.402630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f4682-209b-49eb-ad1c-ee7f761f7664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:36.403263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f4682-209b-49eb-ad1c-ee7f761f7664 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.403549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f4682-209b-49eb-ad1c-ee7f761f7664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.403827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f4682-209b-49eb-ad1c-ee7f761f7664 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.412101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f4682-209b-49eb-ad1c-ee7f761f7664 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:36.413918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f4682-209b-49eb-ad1c-ee7f761f7664 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:36.423691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 5/8] 10.222.0.39 () {58 vars in 1324 bytes} [Tue Jun 7 08:20:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:20:36.429301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69721fdf-d4d5-4263-beea-67e09969dd81 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:36.439466 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69721fdf-d4d5-4263-beea-67e09969dd81 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:36.443965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732a02e-36ff-477c-b848-172d3cff5566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:36.445217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732a02e-36ff-477c-b848-172d3cff5566 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.445217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732a02e-36ff-477c-b848-172d3cff5566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.445456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732a02e-36ff-477c-b848-172d3cff5566 admin admin] PATH_INFO: `/v3/users/d70de4148c8f4696ac913ae91f5a81cb` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.446826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732a02e-36ff-477c-b848-172d3cff5566 admin admin] RBAC: Authorizing `identity:get_user(user_id=d70de4148c8f4696ac913ae91f5a81cb)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:36.448701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732a02e-36ff-477c-b848-172d3cff5566 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:36.450634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 4/9] 10.222.0.39 () {56 vars in 1198 bytes} [Tue Jun 7 08:20:36 2022] GET /identity/v3/users/d70de4148c8f4696ac913ae91f5a81cb => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:38.806326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c3f6d-ff04-42b9-8f89-ab4329b68e9f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.806850 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c3f6d-ff04-42b9-8f89-ab4329b68e9f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.806950 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c3f6d-ff04-42b9-8f89-ab4329b68e9f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.808236 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 6/10] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:38.814235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9700f17-8347-413a-856a-e3798b1c5e0f None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.814732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9700f17-8347-413a-856a-e3798b1c5e0f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.814836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9700f17-8347-413a-856a-e3798b1c5e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.837416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e9700f17-8347-413a-856a-e3798b1c5e0f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:38.838623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e9700f17-8347-413a-856a-e3798b1c5e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:38.847214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6530b5-213d-4d77-8641-e54949a014eb None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.847503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6530b5-213d-4d77-8641-e54949a014eb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.847783 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6530b5-213d-4d77-8641-e54949a014eb None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.849228 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 7/11] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:38.854411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe74cee-7d28-425d-9cc9-cb4781e0537a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.854702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe74cee-7d28-425d-9cc9-cb4781e0537a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.854984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe74cee-7d28-425d-9cc9-cb4781e0537a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.901647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG passlib.handlers.bcrypt [None req-3fe74cee-7d28-425d-9cc9-cb4781e0537a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80228) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 08:20:38.902071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG passlib.handlers.bcrypt [None req-3fe74cee-7d28-425d-9cc9-cb4781e0537a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80228) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 08:20:38.932417 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 5/12] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:20:38.949677 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3fe74cee-7d28-425d-9cc9-cb4781e0537a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:38.951310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3fe74cee-7d28-425d-9cc9-cb4781e0537a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:38.957057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9700f17-8347-413a-856a-e3798b1c5e0f None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:38.983011 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:39.060462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 8/13] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:20:39.070752 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9700f17-8347-413a-856a-e3798b1c5e0f None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:39.083677 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe74cee-7d28-425d-9cc9-cb4781e0537a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:39.109148 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4696d1f-69e2-40ec-a1bf-dd536cdffcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:39.109970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4696d1f-69e2-40ec-a1bf-dd536cdffcdc None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:39.110258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4696d1f-69e2-40ec-a1bf-dd536cdffcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:39.110537 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4696d1f-69e2-40ec-a1bf-dd536cdffcdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:39.122359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4696d1f-69e2-40ec-a1bf-dd536cdffcdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-e4696d1f-69e2-40ec-a1bf-dd536cdffcdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:39.134532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:39.135700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:39.136674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:39.136674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:39.136674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:39.136674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:39.136674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:39.136674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:39.136674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:39.136674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:39.136674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:39.136674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:39.136674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:39.136674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:20:39.136674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 6/14] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:39.142569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4696d1f-69e2-40ec-a1bf-dd536cdffcdc None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:39.197601 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe74cee-7d28-425d-9cc9-cb4781e0537a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:39.200015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:39.232607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:39.238223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4ad16-d7f3-4247-a6c1-0c397f378f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:39.239164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4ad16-d7f3-4247-a6c1-0c397f378f89 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:39.239457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4ad16-d7f3-4247-a6c1-0c397f378f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:39.239741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4ad16-d7f3-4247-a6c1-0c397f378f89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:39.253866 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4696d1f-69e2-40ec-a1bf-dd536cdffcdc None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:39.290717 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a29aae-4b9f-430a-825d-b95382696ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:39.291410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a29aae-4b9f-430a-825d-b95382696ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:39.291697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a29aae-4b9f-430a-825d-b95382696ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:39.291974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a29aae-4b9f-430a-825d-b95382696ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:39.292751 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a29aae-4b9f-430a-825d-b95382696ab8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:39.293057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a29aae-4b9f-430a-825d-b95382696ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:39.294524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a29aae-4b9f-430a-825d-b95382696ab8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:39.307412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 7/15] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:39.313012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4ad16-d7f3-4247-a6c1-0c397f378f89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:39.313799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a29aae-4b9f-430a-825d-b95382696ab8 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:39.314664 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4ad16-d7f3-4247-a6c1-0c397f378f89 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:39.423076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a29aae-4b9f-430a-825d-b95382696ab8 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:39.423407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 9/16] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:20:39.428729 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4ad16-d7f3-4247-a6c1-0c397f378f89 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:39.431674 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:39.462561 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e11a06-1476-4eef-af9d-590d0d92c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:39.463436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e11a06-1476-4eef-af9d-590d0d92c13b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:39.463746 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e11a06-1476-4eef-af9d-590d0d92c13b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:39.464023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e11a06-1476-4eef-af9d-590d0d92c13b None admin] PATH_INFO: `/v3/users/d70de4148c8f4696ac913ae91f5a81cb` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:39.465326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e11a06-1476-4eef-af9d-590d0d92c13b None admin] RBAC: Authorizing `identity:get_user(user_id=d70de4148c8f4696ac913ae91f5a81cb)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:39.466828 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e11a06-1476-4eef-af9d-590d0d92c13b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:39.469143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 8/17] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:20:39 2022] GET /identity/v3/users/d70de4148c8f4696ac913ae91f5a81cb => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:39.474966 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e11a06-1476-4eef-af9d-590d0d92c13b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:39.491985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:39.626712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e11a06-1476-4eef-af9d-590d0d92c13b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:39.662401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4ad16-d7f3-4247-a6c1-0c397f378f89 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:39.672127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9c5d3-3ff6-4d07-b3c3-626408a82ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:39.672944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c5d3-3ff6-4d07-b3c3-626408a82ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:39.673333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c5d3-3ff6-4d07-b3c3-626408a82ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:39.673615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c5d3-3ff6-4d07-b3c3-626408a82ad5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:39.674862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9c5d3-3ff6-4d07-b3c3-626408a82ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:39.676329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9c5d3-3ff6-4d07-b3c3-626408a82ad5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:39.678168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 9/18] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:39.685515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9c5d3-3ff6-4d07-b3c3-626408a82ad5 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:39.706034 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a768b22-12af-4834-ac5c-99e2343fe13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:39.706941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a768b22-12af-4834-ac5c-99e2343fe13d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:39.707227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a768b22-12af-4834-ac5c-99e2343fe13d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:39.707506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a768b22-12af-4834-ac5c-99e2343fe13d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:39.727626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a768b22-12af-4834-ac5c-99e2343fe13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:39.729125 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a768b22-12af-4834-ac5c-99e2343fe13d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-4a768b22-12af-4834-ac5c-99e2343fe13d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:39.750117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:39.751555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:39.752903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:39.752903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:39.752903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:39.752903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:20:39.752903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 10/19] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:39.757587 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a768b22-12af-4834-ac5c-99e2343fe13d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:39.774941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:39.796227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9c5d3-3ff6-4d07-b3c3-626408a82ad5 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:39.820471 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:39.844551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d5c1f-0f77-4dc2-8228-64c267c470c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:39.845252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d5c1f-0f77-4dc2-8228-64c267c470c3 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:39.845541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d5c1f-0f77-4dc2-8228-64c267c470c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:39.845824 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d5c1f-0f77-4dc2-8228-64c267c470c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:39.846671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d5c1f-0f77-4dc2-8228-64c267c470c3 None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.domain.id=default, user.id=d70de4148c8f4696ac913ae91f5a81cb) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:39.846988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d5c1f-0f77-4dc2-8228-64c267c470c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:39.848460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d5c1f-0f77-4dc2-8228-64c267c470c3 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:39.855865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 10/20] 10.222.0.39 () {56 vars in 1322 bytes} [Tue Jun 7 08:20:39 2022] GET /identity/v3/role_assignments?user.id=d70de4148c8f4696ac913ae91f5a81cb&scope.domain.id=default&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 233 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:39.872736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a768b22-12af-4834-ac5c-99e2343fe13d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:39.927813 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851a03a-438c-4cc5-8387-cff83ae2543c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:39.928544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851a03a-438c-4cc5-8387-cff83ae2543c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:39.928832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851a03a-438c-4cc5-8387-cff83ae2543c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:39.929148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851a03a-438c-4cc5-8387-cff83ae2543c None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:39.929925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851a03a-438c-4cc5-8387-cff83ae2543c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:39.930240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851a03a-438c-4cc5-8387-cff83ae2543c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:39.931702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851a03a-438c-4cc5-8387-cff83ae2543c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:39.953796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 11/21] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:42.410928 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618ff0f-d31d-439b-bd19-198a398b9674 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:42.411445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618ff0f-d31d-439b-bd19-198a398b9674 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:42.411517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618ff0f-d31d-439b-bd19-198a398b9674 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:42.413022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 11/22] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:42.418362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ec1e7-b4a8-4718-b0a3-b6d4359499d5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:42.418838 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ec1e7-b4a8-4718-b0a3-b6d4359499d5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:42.418931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ec1e7-b4a8-4718-b0a3-b6d4359499d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:42.441979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6e6ec1e7-b4a8-4718-b0a3-b6d4359499d5 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:42.443541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6e6ec1e7-b4a8-4718-b0a3-b6d4359499d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:42.535942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 12/23] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:20:42.602665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618ff0f-d31d-439b-bd19-198a398b9674 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:42.758326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618ff0f-d31d-439b-bd19-198a398b9674 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:42.814673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9e2d1-4e8d-4a6d-a003-6969826cd666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:42.815477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9e2d1-4e8d-4a6d-a003-6969826cd666 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:42.815765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9e2d1-4e8d-4a6d-a003-6969826cd666 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:42.816047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9e2d1-4e8d-4a6d-a003-6969826cd666 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:42.832094 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9e2d1-4e8d-4a6d-a003-6969826cd666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-d3a9e2d1-4e8d-4a6d-a003-6969826cd666 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:42.840001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:42.840886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:42.841664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:42.841664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:42.841664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:42.841664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:42.841664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:42.841664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:42.841664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:42.841664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:42.841664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:42.841664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:42.841664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:42.841664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:20:42.841664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 12/24] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:42.849238 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ec1e7-b4a8-4718-b0a3-b6d4359499d5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:42.919506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:42.985256 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ec1e7-b4a8-4718-b0a3-b6d4359499d5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:43.028837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e31b09-db0d-4c8e-aee5-2b46d3fe2cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:43.029888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e31b09-db0d-4c8e-aee5-2b46d3fe2cbb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.030472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e31b09-db0d-4c8e-aee5-2b46d3fe2cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.030957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e31b09-db0d-4c8e-aee5-2b46d3fe2cbb None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.032115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e31b09-db0d-4c8e-aee5-2b46d3fe2cbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:43.032632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e31b09-db0d-4c8e-aee5-2b46d3fe2cbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:43.034258 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e31b09-db0d-4c8e-aee5-2b46d3fe2cbb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:43.052823 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 13/25] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:43.062870 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9e2d1-4e8d-4a6d-a003-6969826cd666 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:43.081655 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:43.237432 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9e2d1-4e8d-4a6d-a003-6969826cd666 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:43.287622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da41ddc-8f27-40b6-bd92-1910accb2558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:43.288423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da41ddc-8f27-40b6-bd92-1910accb2558 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.288709 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da41ddc-8f27-40b6-bd92-1910accb2558 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.288991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da41ddc-8f27-40b6-bd92-1910accb2558 None admin] PATH_INFO: `/v3/users/d70de4148c8f4696ac913ae91f5a81cb` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.290308 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da41ddc-8f27-40b6-bd92-1910accb2558 None admin] RBAC: Authorizing `identity:get_user(user_id=d70de4148c8f4696ac913ae91f5a81cb)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:43.291778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da41ddc-8f27-40b6-bd92-1910accb2558 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:43.293466 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 13/26] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:20:43 2022] GET /identity/v3/users/d70de4148c8f4696ac913ae91f5a81cb => generated 247 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:43.299611 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e31b09-db0d-4c8e-aee5-2b46d3fe2cbb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:43.378225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43868b-9b32-4267-a628-478abfccc34a None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.378800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43868b-9b32-4267-a628-478abfccc34a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.379277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43868b-9b32-4267-a628-478abfccc34a None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.380856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 14/27] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:43.388404 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4c65-e481-4fd6-bc60-d0b33e01dacd None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.388985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4c65-e481-4fd6-bc60-d0b33e01dacd None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.389487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4c65-e481-4fd6-bc60-d0b33e01dacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.423175 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d70d4c65-e481-4fd6-bc60-d0b33e01dacd None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:43.425200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d70d4c65-e481-4fd6-bc60-d0b33e01dacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:43.489770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e31b09-db0d-4c8e-aee5-2b46d3fe2cbb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:43.515473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:43.542957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcab717-dd9a-465a-917a-8cccd9f4f591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:43.543990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcab717-dd9a-465a-917a-8cccd9f4f591 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.544420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcab717-dd9a-465a-917a-8cccd9f4f591 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.544843 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcab717-dd9a-465a-917a-8cccd9f4f591 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.547120 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcab717-dd9a-465a-917a-8cccd9f4f591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:43.548987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcab717-dd9a-465a-917a-8cccd9f4f591 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:43.550958 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 14/28] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:43.552436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55f850-2f6d-4883-a9a1-c7b1ff0fcb95 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.552971 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55f850-2f6d-4883-a9a1-c7b1ff0fcb95 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.553447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55f850-2f6d-4883-a9a1-c7b1ff0fcb95 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.555391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 15/29] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:43.556863 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b0bc7-170b-450f-bd39-28c579c440f6 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.557603 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b0bc7-170b-450f-bd39-28c579c440f6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.558061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b0bc7-170b-450f-bd39-28c579c440f6 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.560289 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 16/30] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:43.561295 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b0bc7-170b-450f-bd39-28c579c440f6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:43.575961 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 15/31] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:20:43.579081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63247ba7-241c-4b0e-982c-e6c4917f035c None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.579944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63247ba7-241c-4b0e-982c-e6c4917f035c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.580721 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63247ba7-241c-4b0e-982c-e6c4917f035c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.589799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:43.620007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-63247ba7-241c-4b0e-982c-e6c4917f035c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:43.622267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-63247ba7-241c-4b0e-982c-e6c4917f035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:43.789307 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b0bc7-170b-450f-bd39-28c579c440f6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:43.793546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 16/32] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:43.795550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df07ca-1019-47a3-88da-3d13d948766f None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.795550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df07ca-1019-47a3-88da-3d13d948766f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.795887 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df07ca-1019-47a3-88da-3d13d948766f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.830208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e0df07ca-1019-47a3-88da-3d13d948766f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:43.831421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e0df07ca-1019-47a3-88da-3d13d948766f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:43.867023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3c27e-8fb4-4ea6-8c4d-ea6a8ee80bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:43.867920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3c27e-8fb4-4ea6-8c4d-ea6a8ee80bc8 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.868204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3c27e-8fb4-4ea6-8c4d-ea6a8ee80bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.868484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3c27e-8fb4-4ea6-8c4d-ea6a8ee80bc8 None admin] PATH_INFO: `/v3/domains/default/users/d70de4148c8f4696ac913ae91f5a81cb/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.872960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3c27e-8fb4-4ea6-8c4d-ea6a8ee80bc8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d70de4148c8f4696ac913ae91f5a81cb, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:43.874503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3c27e-8fb4-4ea6-8c4d-ea6a8ee80bc8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:43.989185 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 17/33] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:43.990496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df07ca-1019-47a3-88da-3d13d948766f None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:44.081197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 17/34] 10.222.0.39 () {58 vars in 1361 bytes} [Tue Jun 7 08:20:43 2022] PUT /identity/v3/domains/default/users/d70de4148c8f4696ac913ae91f5a81cb/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:20:44.082846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3c27e-8fb4-4ea6-8c4d-ea6a8ee80bc8 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:44.206318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:44.238216 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:44.338919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df07ca-1019-47a3-88da-3d13d948766f None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:44.379362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b676d5c-c220-4086-9d84-ce91bbb19658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:44.380467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b676d5c-c220-4086-9d84-ce91bbb19658 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.380753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b676d5c-c220-4086-9d84-ce91bbb19658 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.381031 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b676d5c-c220-4086-9d84-ce91bbb19658 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:44.381821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b676d5c-c220-4086-9d84-ce91bbb19658 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:44.383256 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b676d5c-c220-4086-9d84-ce91bbb19658 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:44.405218 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3c27e-8fb4-4ea6-8c4d-ea6a8ee80bc8 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:44.486110 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d45345-4f53-4cd1-83a6-94b267288a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:44.486819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d45345-4f53-4cd1-83a6-94b267288a7c None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.487112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d45345-4f53-4cd1-83a6-94b267288a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.487395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d45345-4f53-4cd1-83a6-94b267288a7c None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:44.488170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d45345-4f53-4cd1-83a6-94b267288a7c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:44.489839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d45345-4f53-4cd1-83a6-94b267288a7c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:44.516591 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 18/35] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Jun 7 08:20:44 2022] POST /identity/v3/roles => generated 225 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:44.517755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d45345-4f53-4cd1-83a6-94b267288a7c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:44.541912 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 18/36] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Jun 7 08:20:43 2022] POST /identity/v3/roles => generated 219 bytes in 552 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:44.543666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38e88b-4113-4893-96f0-37de91a229c2 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.543963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38e88b-4113-4893-96f0-37de91a229c2 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.544272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38e88b-4113-4893-96f0-37de91a229c2 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:44.546309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 19/37] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:44.565118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2592762-1f53-49a0-a7f4-798daf170129 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.565118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2592762-1f53-49a0-a7f4-798daf170129 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.565118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2592762-1f53-49a0-a7f4-798daf170129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:44.608529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-f2592762-1f53-49a0-a7f4-798daf170129 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:44.610559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f2592762-1f53-49a0-a7f4-798daf170129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:44.769403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 20/38] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:44.824979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2592762-1f53-49a0-a7f4-798daf170129 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:44.830938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:44.852778 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d45345-4f53-4cd1-83a6-94b267288a7c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:44.925302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237b75e-9064-41cb-839c-bb5e4b6cac42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:44.926731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237b75e-9064-41cb-839c-bb5e4b6cac42 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.927023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237b75e-9064-41cb-839c-bb5e4b6cac42 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.927304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237b75e-9064-41cb-839c-bb5e4b6cac42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:44.946117 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:44.950568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237b75e-9064-41cb-839c-bb5e4b6cac42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:44.953983 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237b75e-9064-41cb-839c-bb5e4b6cac42 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-e237b75e-9064-41cb-839c-bb5e4b6cac42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:44.981890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:44.982790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:44.984124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:44.984124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:44.984124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:44.984124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:20:44.985270 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 19/39] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:44.986586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995093a-bc21-4c68-9466-b0cf28afbb5a None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.986928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995093a-bc21-4c68-9466-b0cf28afbb5a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.987208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995093a-bc21-4c68-9466-b0cf28afbb5a None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:44.988759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 20/40] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:44.997240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5de16d-a7e9-4ba0-851d-9fb94bd65448 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.997816 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5de16d-a7e9-4ba0-851d-9fb94bd65448 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.998128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5de16d-a7e9-4ba0-851d-9fb94bd65448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:45.019826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2592762-1f53-49a0-a7f4-798daf170129 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:45.028670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6d5de16d-a7e9-4ba0-851d-9fb94bd65448 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:45.032114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6d5de16d-a7e9-4ba0-851d-9fb94bd65448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:45.068338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85d0e0-4157-497c-b0f1-9c781e19f192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:45.069856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85d0e0-4157-497c-b0f1-9c781e19f192 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:45.070165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85d0e0-4157-497c-b0f1-9c781e19f192 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:45.070629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85d0e0-4157-497c-b0f1-9c781e19f192 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:45.072744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85d0e0-4157-497c-b0f1-9c781e19f192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:45.075330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85d0e0-4157-497c-b0f1-9c781e19f192 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:45.083120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 21/41] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:45.083120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85d0e0-4157-497c-b0f1-9c781e19f192 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:45.129274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 21/42] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:20:45.131327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5de16d-a7e9-4ba0-851d-9fb94bd65448 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:45.238086 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85d0e0-4157-497c-b0f1-9c781e19f192 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:45.262627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:45.274957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:45.287980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7aa638-23a3-460f-99e4-97ec99113cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:45.288760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7aa638-23a3-460f-99e4-97ec99113cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:45.288950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7aa638-23a3-460f-99e4-97ec99113cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:45.289273 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7aa638-23a3-460f-99e4-97ec99113cf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:45.290018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7aa638-23a3-460f-99e4-97ec99113cf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:45.290331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7aa638-23a3-460f-99e4-97ec99113cf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:45.291765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7aa638-23a3-460f-99e4-97ec99113cf5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:45.316563 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 22/43] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:45.316563 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7aa638-23a3-460f-99e4-97ec99113cf5 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:45.321876 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5de16d-a7e9-4ba0-851d-9fb94bd65448 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:45.363814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e446cf1-79c8-4462-84b6-3893700ed6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:45.364743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e446cf1-79c8-4462-84b6-3893700ed6a7 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:45.365229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e446cf1-79c8-4462-84b6-3893700ed6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:45.365639 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e446cf1-79c8-4462-84b6-3893700ed6a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:45.366493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e446cf1-79c8-4462-84b6-3893700ed6a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:45.368797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e446cf1-79c8-4462-84b6-3893700ed6a7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:45.409492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 22/44] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:20:45 2022] POST /identity/v3/projects => generated 312 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:45.411918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e446cf1-79c8-4462-84b6-3893700ed6a7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:45.462320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7aa638-23a3-460f-99e4-97ec99113cf5 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:45.515508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc1972-d7a3-400b-9ea8-d72708f5dc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:45.518348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc1972-d7a3-400b-9ea8-d72708f5dc9e None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:45.519546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc1972-d7a3-400b-9ea8-d72708f5dc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:45.520649 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc1972-d7a3-400b-9ea8-d72708f5dc9e None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:45.523010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc1972-d7a3-400b-9ea8-d72708f5dc9e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:45.527045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc1972-d7a3-400b-9ea8-d72708f5dc9e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:45.559938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 23/45] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Jun 7 08:20:45 2022] POST /identity/v3/roles => generated 223 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:45.596522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e446cf1-79c8-4462-84b6-3893700ed6a7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:45.637528 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:45.646325 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72f873-77d8-42c9-9c01-d82e65ad3a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:45.647448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72f873-77d8-42c9-9c01-d82e65ad3a34 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:45.647977 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72f873-77d8-42c9-9c01-d82e65ad3a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:45.648444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72f873-77d8-42c9-9c01-d82e65ad3a34 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:45.649265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72f873-77d8-42c9-9c01-d82e65ad3a34 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:45.650890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72f873-77d8-42c9-9c01-d82e65ad3a34 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:45.693341 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 23/46] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:20:45 2022] POST /identity/v3/projects => generated 301 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:46.950287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abbe2e-40a5-4cfd-a00c-136db7d6649c None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:46.951352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abbe2e-40a5-4cfd-a00c-136db7d6649c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:46.951352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abbe2e-40a5-4cfd-a00c-136db7d6649c None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:46.952374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 24/47] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:46.959170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c51a5-99ba-4ecd-9a24-dea365b22360 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:46.959622 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c51a5-99ba-4ecd-9a24-dea365b22360 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:46.959811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c51a5-99ba-4ecd-9a24-dea365b22360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:46.979020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-456c51a5-99ba-4ecd-9a24-dea365b22360 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:46.980218 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-456c51a5-99ba-4ecd-9a24-dea365b22360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:47.049947 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 24/48] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:47.073815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abbe2e-40a5-4cfd-a00c-136db7d6649c None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.119602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:47.187364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abbe2e-40a5-4cfd-a00c-136db7d6649c None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.223315 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6db33-f1ef-49fa-82bf-cb5a4c26bbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.224109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6db33-f1ef-49fa-82bf-cb5a4c26bbba None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.224397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6db33-f1ef-49fa-82bf-cb5a4c26bbba None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.224679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6db33-f1ef-49fa-82bf-cb5a4c26bbba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.237194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6db33-f1ef-49fa-82bf-cb5a4c26bbba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-83a6db33-f1ef-49fa-82bf-cb5a4c26bbba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:47.244798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:47.245935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:47.246907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:47.246907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:47.246907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:47.246907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:47.246907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:47.246907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:47.246907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:47.246907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:47.246907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:47.246907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:47.246907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:47.246907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:20:47.246907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 25/49] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:47.252539 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c51a5-99ba-4ecd-9a24-dea365b22360 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.354188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c51a5-99ba-4ecd-9a24-dea365b22360 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.368121 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:47.398443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387687a-8fcd-4e73-ae55-f2c8701d3e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.399138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387687a-8fcd-4e73-ae55-f2c8701d3e74 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.399402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387687a-8fcd-4e73-ae55-f2c8701d3e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.399679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387687a-8fcd-4e73-ae55-f2c8701d3e74 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.400425 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387687a-8fcd-4e73-ae55-f2c8701d3e74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:47.400734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387687a-8fcd-4e73-ae55-f2c8701d3e74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.402136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387687a-8fcd-4e73-ae55-f2c8701d3e74 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.411764 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 25/50] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:47.417349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6db33-f1ef-49fa-82bf-cb5a4c26bbba None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.533113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:47.536554 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6db33-f1ef-49fa-82bf-cb5a4c26bbba None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.581783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72668cb-c45a-4b37-bba4-d3db42a7b2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.582741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72668cb-c45a-4b37-bba4-d3db42a7b2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.583082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72668cb-c45a-4b37-bba4-d3db42a7b2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.583420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72668cb-c45a-4b37-bba4-d3db42a7b2d8 None admin] PATH_INFO: `/v3/users/d70de4148c8f4696ac913ae91f5a81cb` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.584758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72668cb-c45a-4b37-bba4-d3db42a7b2d8 None admin] RBAC: Authorizing `identity:get_user(user_id=d70de4148c8f4696ac913ae91f5a81cb)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.586396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72668cb-c45a-4b37-bba4-d3db42a7b2d8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.588069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 26/51] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/users/d70de4148c8f4696ac913ae91f5a81cb => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:47.594157 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387687a-8fcd-4e73-ae55-f2c8701d3e74 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.695714 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387687a-8fcd-4e73-ae55-f2c8701d3e74 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.730673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e74ed-96d7-4625-8384-35ab5d175679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.731476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e74ed-96d7-4625-8384-35ab5d175679 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.731764 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e74ed-96d7-4625-8384-35ab5d175679 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.732045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e74ed-96d7-4625-8384-35ab5d175679 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.733267 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e74ed-96d7-4625-8384-35ab5d175679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.735572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e74ed-96d7-4625-8384-35ab5d175679 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.737283 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 26/52] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:47.742948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72668cb-c45a-4b37-bba4-d3db42a7b2d8 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.857017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72668cb-c45a-4b37-bba4-d3db42a7b2d8 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.889814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2617d28-4aea-4fbe-993f-fa9e82899851 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.890431 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2617d28-4aea-4fbe-993f-fa9e82899851 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.890915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2617d28-4aea-4fbe-993f-fa9e82899851 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.892728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 27/53] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:47.899864 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164911a5-f5b7-4ab8-8851-3ef6b958bbb0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.899960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164911a5-f5b7-4ab8-8851-3ef6b958bbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.900383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164911a5-f5b7-4ab8-8851-3ef6b958bbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.905929 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b04d6-f315-40c8-bd6f-f59e2fdc89bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.906917 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b04d6-f315-40c8-bd6f-f59e2fdc89bb None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.907266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b04d6-f315-40c8-bd6f-f59e2fdc89bb None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.907609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b04d6-f315-40c8-bd6f-f59e2fdc89bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.908516 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b04d6-f315-40c8-bd6f-f59e2fdc89bb None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.domain.id=default, user.id=d70de4148c8f4696ac913ae91f5a81cb) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:47.908894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b04d6-f315-40c8-bd6f-f59e2fdc89bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.910420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b04d6-f315-40c8-bd6f-f59e2fdc89bb None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.917185 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 27/54] 10.222.0.39 () {56 vars in 1322 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/role_assignments?user.id=d70de4148c8f4696ac913ae91f5a81cb&scope.domain.id=default&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 530 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:47.920254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-164911a5-f5b7-4ab8-8851-3ef6b958bbb0 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:47.922453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-164911a5-f5b7-4ab8-8851-3ef6b958bbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:47.967095 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:48.007707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 28/55] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:20:48.034536 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b04d6-f315-40c8-bd6f-f59e2fdc89bb None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:48.072705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:48.160150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b04d6-f315-40c8-bd6f-f59e2fdc89bb None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:48.205310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66089173-d202-4b1d-bef9-de40f0015901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:48.206125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66089173-d202-4b1d-bef9-de40f0015901 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:48.206415 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66089173-d202-4b1d-bef9-de40f0015901 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:48.206694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66089173-d202-4b1d-bef9-de40f0015901 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:48.207916 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66089173-d202-4b1d-bef9-de40f0015901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:48.209467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66089173-d202-4b1d-bef9-de40f0015901 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:48.211138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 28/56] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:48.217001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164911a5-f5b7-4ab8-8851-3ef6b958bbb0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:48.324652 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164911a5-f5b7-4ab8-8851-3ef6b958bbb0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:48.362931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e024674-68de-4a05-a164-67483156ef9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:48.363606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e024674-68de-4a05-a164-67483156ef9b None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:48.363888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e024674-68de-4a05-a164-67483156ef9b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:48.364164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e024674-68de-4a05-a164-67483156ef9b None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:48.364872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e024674-68de-4a05-a164-67483156ef9b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:48.366358 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e024674-68de-4a05-a164-67483156ef9b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:48.393474 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 29/57] 10.222.0.39 () {60 vars in 1140 bytes} [Tue Jun 7 08:20:48 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:50.720567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d37084-f730-4abc-bda0-04facea65642 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:50.721117 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d37084-f730-4abc-bda0-04facea65642 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:50.721203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d37084-f730-4abc-bda0-04facea65642 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:50.722512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 29/58] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:50.727871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223be33-6e83-432f-9c5c-df16506ec8e5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:50.728271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223be33-6e83-432f-9c5c-df16506ec8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:50.728438 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223be33-6e83-432f-9c5c-df16506ec8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:50.745760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0223be33-6e83-432f-9c5c-df16506ec8e5 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:50.747437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0223be33-6e83-432f-9c5c-df16506ec8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:50.764639 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:50.816256 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 30/59] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:50.840782 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d37084-f730-4abc-bda0-04facea65642 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:50.940747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:50.944892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d37084-f730-4abc-bda0-04facea65642 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:50.978382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904a438-77e3-48a7-8f6b-7e01391d04af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:50.979094 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904a438-77e3-48a7-8f6b-7e01391d04af None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:50.979378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904a438-77e3-48a7-8f6b-7e01391d04af None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:50.979653 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904a438-77e3-48a7-8f6b-7e01391d04af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:50.980820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904a438-77e3-48a7-8f6b-7e01391d04af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:50.982293 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904a438-77e3-48a7-8f6b-7e01391d04af None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:50.984761 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 30/60] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:50.990595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223be33-6e83-432f-9c5c-df16506ec8e5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:51.091015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223be33-6e83-432f-9c5c-df16506ec8e5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:51.125151 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779a1d3-7cf7-4ddc-b7ae-243b96572c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:51.125829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779a1d3-7cf7-4ddc-b7ae-243b96572c31 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:51.126114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779a1d3-7cf7-4ddc-b7ae-243b96572c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:51.126394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779a1d3-7cf7-4ddc-b7ae-243b96572c31 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:51.127119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779a1d3-7cf7-4ddc-b7ae-243b96572c31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:51.128567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779a1d3-7cf7-4ddc-b7ae-243b96572c31 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:51.153394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 31/61] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:20:50 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:54.494954 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f91e5-bf57-4ce2-af00-5fa37ba6b9e7 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:54.495856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f91e5-bf57-4ce2-af00-5fa37ba6b9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:54.496279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f91e5-bf57-4ce2-af00-5fa37ba6b9e7 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:54.498162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 31/62] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:54.504650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea5947-b817-4d8b-96bc-a27551f802fe None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:54.505503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea5947-b817-4d8b-96bc-a27551f802fe None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:54.505951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea5947-b817-4d8b-96bc-a27551f802fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:54.529512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e1ea5947-b817-4d8b-96bc-a27551f802fe None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:54.531283 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e1ea5947-b817-4d8b-96bc-a27551f802fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:54.538117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:54.635732 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 32/63] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:54.664912 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f91e5-bf57-4ce2-af00-5fa37ba6b9e7 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:54.740436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a27091-5fbf-4ffd-ace8-c58d1d6aefb9 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:54.740959 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a27091-5fbf-4ffd-ace8-c58d1d6aefb9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:54.741435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a27091-5fbf-4ffd-ace8-c58d1d6aefb9 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:54.743396 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 33/64] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:54.751322 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce4ad0-8cd4-4927-a22c-eecf87d38836 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:54.751964 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce4ad0-8cd4-4927-a22c-eecf87d38836 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:54.752402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce4ad0-8cd4-4927-a22c-eecf87d38836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:54.783693 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-43ce4ad0-8cd4-4927-a22c-eecf87d38836 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:54.789200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-43ce4ad0-8cd4-4927-a22c-eecf87d38836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:54.804768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f91e5-bf57-4ce2-af00-5fa37ba6b9e7 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:54.846423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4d505-ac6f-4fac-aa6f-c054c725bd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:54.847461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4d505-ac6f-4fac-aa6f-c054c725bd9a None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:54.848565 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4d505-ac6f-4fac-aa6f-c054c725bd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:54.848994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4d505-ac6f-4fac-aa6f-c054c725bd9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:54.861074 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4d505-ac6f-4fac-aa6f-c054c725bd9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-bee4d505-ac6f-4fac-aa6f-c054c725bd9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:54.869903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:54.870991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:54.871919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:54.871919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:54.871919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:54.871919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:54.871919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:54.871919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:54.871919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:54.871919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:54.871919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:54.871919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:54.871919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:20:54.871919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:20:54.875166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 32/65] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:54.883259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4d505-ac6f-4fac-aa6f-c054c725bd9a None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:54.956104 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:54.970613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 34/66] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:55.038639 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce4ad0-8cd4-4927-a22c-eecf87d38836 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:55.083546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4d505-ac6f-4fac-aa6f-c054c725bd9a None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:55.136124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e95115-b42b-4cb1-9180-763a626cd235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:55.137323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e95115-b42b-4cb1-9180-763a626cd235 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.137909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e95115-b42b-4cb1-9180-763a626cd235 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.138359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e95115-b42b-4cb1-9180-763a626cd235 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.139938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e95115-b42b-4cb1-9180-763a626cd235 None admin] RBAC: Adding query filter params (name=member) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:55.140418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e95115-b42b-4cb1-9180-763a626cd235 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:55.142817 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e95115-b42b-4cb1-9180-763a626cd235 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:55.150129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:55.154266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 33/67] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:20:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:55.156423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13131993-78c9-47df-94f0-7504ee4a1242 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.156720 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13131993-78c9-47df-94f0-7504ee4a1242 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.156994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13131993-78c9-47df-94f0-7504ee4a1242 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.158499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 34/68] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:55.164936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13131993-78c9-47df-94f0-7504ee4a1242 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:55.198013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce4ad0-8cd4-4927-a22c-eecf87d38836 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:55.244996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ff39a-6514-4be0-9052-e2e2aaf2f7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:55.246645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ff39a-6514-4be0-9052-e2e2aaf2f7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.247107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ff39a-6514-4be0-9052-e2e2aaf2f7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.247559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ff39a-6514-4be0-9052-e2e2aaf2f7b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.263435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ff39a-6514-4be0-9052-e2e2aaf2f7b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:55.270901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-8d0ff39a-6514-4be0-9052-e2e2aaf2f7b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:55.275497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:55.276558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:55.277452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:55.277452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:55.277452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:55.277452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:55.277452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:55.277452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:55.277452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:55.277452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:55.277452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:55.277452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:55.277452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:55.277452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:20:55.283585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 35/69] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:55.283585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53aa77-810a-4042-a7c2-005a4e32d557 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.283585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53aa77-810a-4042-a7c2-005a4e32d557 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.283585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53aa77-810a-4042-a7c2-005a4e32d557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.297570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13131993-78c9-47df-94f0-7504ee4a1242 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:55.304680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-bc53aa77-810a-4042-a7c2-005a4e32d557 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:55.305929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bc53aa77-810a-4042-a7c2-005a4e32d557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:55.339111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dcd57-248c-4a03-81b6-861344a12db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:55.339984 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dcd57-248c-4a03-81b6-861344a12db7 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.340275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dcd57-248c-4a03-81b6-861344a12db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.340548 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dcd57-248c-4a03-81b6-861344a12db7 None admin] PATH_INFO: `/v3/users/d49ddf0ef68c45e4a61eddb790691a24` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.355249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dcd57-248c-4a03-81b6-861344a12db7 None admin] RBAC: Authorizing `identity:get_user(user_id=d49ddf0ef68c45e4a61eddb790691a24)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:55.356627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dcd57-248c-4a03-81b6-861344a12db7 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:55.361324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 35/70] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:20:55 2022] GET /identity/v3/users/d49ddf0ef68c45e4a61eddb790691a24 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:55.361324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dcd57-248c-4a03-81b6-861344a12db7 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:55.393236 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 36/71] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:55.394743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d681c21-54ae-4c51-93cb-caacb39fe537 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.395043 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d681c21-54ae-4c51-93cb-caacb39fe537 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.395326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d681c21-54ae-4c51-93cb-caacb39fe537 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.396678 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 37/72] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:55.397566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d681c21-54ae-4c51-93cb-caacb39fe537 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:55.468916 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:55.482992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dcd57-248c-4a03-81b6-861344a12db7 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:55.516429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d681c21-54ae-4c51-93cb-caacb39fe537 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:55.520329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d6b62-5e44-4ea5-8721-6ca9823c1bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:55.521003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d6b62-5e44-4ea5-8721-6ca9823c1bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.521317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d6b62-5e44-4ea5-8721-6ca9823c1bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.521597 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d6b62-5e44-4ea5-8721-6ca9823c1bb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.522348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d6b62-5e44-4ea5-8721-6ca9823c1bb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:55.522657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d6b62-5e44-4ea5-8721-6ca9823c1bb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:55.524040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d6b62-5e44-4ea5-8721-6ca9823c1bb9 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:55.533167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 36/73] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:55.534481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2acf2b-db48-4f1a-a711-a255c4b007d2 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.534771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2acf2b-db48-4f1a-a711-a255c4b007d2 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.535046 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2acf2b-db48-4f1a-a711-a255c4b007d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.552850 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-be2acf2b-db48-4f1a-a711-a255c4b007d2 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:55.553997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-be2acf2b-db48-4f1a-a711-a255c4b007d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:55.557060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:55.561928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02359d92-038e-49fc-8e98-e1ca5ecf7576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:55.562890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02359d92-038e-49fc-8e98-e1ca5ecf7576 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.563306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02359d92-038e-49fc-8e98-e1ca5ecf7576 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.563598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02359d92-038e-49fc-8e98-e1ca5ecf7576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.632773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 37/74] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:55.637361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2acf2b-db48-4f1a-a711-a255c4b007d2 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:55.647168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02359d92-038e-49fc-8e98-e1ca5ecf7576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:55.648960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02359d92-038e-49fc-8e98-e1ca5ecf7576 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:55.711462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:55.739359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2acf2b-db48-4f1a-a711-a255c4b007d2 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:55.753759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 38/75] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:55.754625 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02359d92-038e-49fc-8e98-e1ca5ecf7576 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:55.775312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024793c-1c32-442b-a0c5-6ffb3fed3d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:55.776113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024793c-1c32-442b-a0c5-6ffb3fed3d71 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.776403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024793c-1c32-442b-a0c5-6ffb3fed3d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.776681 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024793c-1c32-442b-a0c5-6ffb3fed3d71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.777911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024793c-1c32-442b-a0c5-6ffb3fed3d71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:55.779381 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024793c-1c32-442b-a0c5-6ffb3fed3d71 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:55.781065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 38/76] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:55.782377 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105d1aa-526d-48e9-821a-78620a82c2ff None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.782672 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105d1aa-526d-48e9-821a-78620a82c2ff None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.782955 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105d1aa-526d-48e9-821a-78620a82c2ff None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.784216 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 39/77] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:55.784918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105d1aa-526d-48e9-821a-78620a82c2ff None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:55.796603 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:55.873244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02359d92-038e-49fc-8e98-e1ca5ecf7576 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:55.904026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105d1aa-526d-48e9-821a-78620a82c2ff None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:55.913431 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a2ea7-070e-4fa0-b4b9-207c08183457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:55.914312 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a2ea7-070e-4fa0-b4b9-207c08183457 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.914601 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a2ea7-070e-4fa0-b4b9-207c08183457 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.914885 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a2ea7-070e-4fa0-b4b9-207c08183457 None admin] PATH_INFO: `/v3/users/d70de4148c8f4696ac913ae91f5a81cb` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.916112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a2ea7-070e-4fa0-b4b9-207c08183457 None admin] RBAC: Authorizing `identity:get_user(user_id=d70de4148c8f4696ac913ae91f5a81cb)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:55.917597 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a2ea7-070e-4fa0-b4b9-207c08183457 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:55.923871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 39/78] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:20:55 2022] GET /identity/v3/users/d70de4148c8f4696ac913ae91f5a81cb => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:55.923871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a2ea7-070e-4fa0-b4b9-207c08183457 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:55.942567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895f3b1-886a-4c79-a89e-50aea9633e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:55.943384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895f3b1-886a-4c79-a89e-50aea9633e1f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.943669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895f3b1-886a-4c79-a89e-50aea9633e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.943946 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895f3b1-886a-4c79-a89e-50aea9633e1f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.957039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895f3b1-886a-4c79-a89e-50aea9633e1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:55.957895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-8895f3b1-886a-4c79-a89e-50aea9633e1f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:55.966697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:55.967890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 40/79] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:20:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:55.973508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895f3b1-886a-4c79-a89e-50aea9633e1f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.032751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a2ea7-070e-4fa0-b4b9-207c08183457 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.040805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:56.068717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60747af0-c7d0-46e3-98bb-e02e5c10d166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.069629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747af0-c7d0-46e3-98bb-e02e5c10d166 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.069917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747af0-c7d0-46e3-98bb-e02e5c10d166 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.070194 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747af0-c7d0-46e3-98bb-e02e5c10d166 None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.071425 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60747af0-c7d0-46e3-98bb-e02e5c10d166 None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:56.072880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60747af0-c7d0-46e3-98bb-e02e5c10d166 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:56.074658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 40/80] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:20:55 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:56.075841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cb970-8ea7-470f-be9e-e9d2ad3714ea None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.076187 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cb970-8ea7-470f-be9e-e9d2ad3714ea None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.076449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cb970-8ea7-470f-be9e-e9d2ad3714ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.081503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895f3b1-886a-4c79-a89e-50aea9633e1f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.097315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-cc9cb970-8ea7-470f-be9e-e9d2ad3714ea None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:56.098563 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cc9cb970-8ea7-470f-be9e-e9d2ad3714ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:56.122374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81d1cc-0c2c-4a8d-ba41-b06ffa7e11b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.123081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81d1cc-0c2c-4a8d-ba41-b06ffa7e11b8 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.123498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81d1cc-0c2c-4a8d-ba41-b06ffa7e11b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.123662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81d1cc-0c2c-4a8d-ba41-b06ffa7e11b8 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.124460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81d1cc-0c2c-4a8d-ba41-b06ffa7e11b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:56.126031 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81d1cc-0c2c-4a8d-ba41-b06ffa7e11b8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:56.157770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 41/81] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:20:55 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:56.158720 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81d1cc-0c2c-4a8d-ba41-b06ffa7e11b8 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.205118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 41/82] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:56.205118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9cb970-8ea7-470f-be9e-e9d2ad3714ea None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.251514 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:56.317176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81d1cc-0c2c-4a8d-ba41-b06ffa7e11b8 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.341187 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9cb970-8ea7-470f-be9e-e9d2ad3714ea None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.350287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:56.364534 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e9707-f38f-4a39-aff4-db50daf4c2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.365101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e9707-f38f-4a39-aff4-db50daf4c2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.366478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e9707-f38f-4a39-aff4-db50daf4c2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.367070 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e9707-f38f-4a39-aff4-db50daf4c2f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.388889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be060860-4235-43b3-b318-d2b16f2cbcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.389748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be060860-4235-43b3-b318-d2b16f2cbcad None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.390096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be060860-4235-43b3-b318-d2b16f2cbcad None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.390316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be060860-4235-43b3-b318-d2b16f2cbcad None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.391195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be060860-4235-43b3-b318-d2b16f2cbcad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:56.391512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be060860-4235-43b3-b318-d2b16f2cbcad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:56.392962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be060860-4235-43b3-b318-d2b16f2cbcad None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:56.403177 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 42/83] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:56.404136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be060860-4235-43b3-b318-d2b16f2cbcad None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.457219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e9707-f38f-4a39-aff4-db50daf4c2f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:56.459234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e9707-f38f-4a39-aff4-db50daf4c2f8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:56.520241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:56.536702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be060860-4235-43b3-b318-d2b16f2cbcad None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.587030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089434b2-7bf8-4a34-a638-7626d8a16c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.587711 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089434b2-7bf8-4a34-a638-7626d8a16c42 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.587996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089434b2-7bf8-4a34-a638-7626d8a16c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.588270 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089434b2-7bf8-4a34-a638-7626d8a16c42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.589142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089434b2-7bf8-4a34-a638-7626d8a16c42 None admin] RBAC: Adding query filter params (role.id=2bdabfafa05e443e842fe34d5f03e2c9, scope.project.id=d00b803196664866a4fd2eb970b1be0b, user.id=d49ddf0ef68c45e4a61eddb790691a24) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:56.589761 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089434b2-7bf8-4a34-a638-7626d8a16c42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:56.591002 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089434b2-7bf8-4a34-a638-7626d8a16c42 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:56.600635 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 43/84] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/role_assignments?user.id=d49ddf0ef68c45e4a61eddb790691a24&scope.project.id=d00b803196664866a4fd2eb970b1be0b&role.id=2bdabfafa05e443e842fe34d5f03e2c9 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:56.601850 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089434b2-7bf8-4a34-a638-7626d8a16c42 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.616965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 42/85] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:56.618612 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e9707-f38f-4a39-aff4-db50daf4c2f8 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.648261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:56.725176 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089434b2-7bf8-4a34-a638-7626d8a16c42 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.740539 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e9707-f38f-4a39-aff4-db50daf4c2f8 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.765953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ebbd8-67e6-423f-b599-f90b4a92a837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.766889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ebbd8-67e6-423f-b599-f90b4a92a837 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.767179 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ebbd8-67e6-423f-b599-f90b4a92a837 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.767467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ebbd8-67e6-423f-b599-f90b4a92a837 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.780726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ebbd8-67e6-423f-b599-f90b4a92a837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:56.782878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125d3a7-37f4-40e1-811b-16a555a62a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.784026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125d3a7-37f4-40e1-811b-16a555a62a0f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.784530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125d3a7-37f4-40e1-811b-16a555a62a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.785036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125d3a7-37f4-40e1-811b-16a555a62a0f None admin] PATH_INFO: `/v3/users/d49ddf0ef68c45e4a61eddb790691a24` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.786724 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125d3a7-37f4-40e1-811b-16a555a62a0f None admin] RBAC: Authorizing `identity:get_user(user_id=d49ddf0ef68c45e4a61eddb790691a24)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:56.788526 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125d3a7-37f4-40e1-811b-16a555a62a0f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-1f7ebbd8-67e6-423f-b599-f90b4a92a837 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:56.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:56.789923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:56.790676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:56.790676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:56.790676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:56.790676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:56.790676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:56.790676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:56.790676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:56.790676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:56.790676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:56.790676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:56.790676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:20:56.790676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:20:56.790676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 44/86] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:56.791318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ebbd8-67e6-423f-b599-f90b4a92a837 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.793793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 43/87] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/users/d49ddf0ef68c45e4a61eddb790691a24 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:56.802152 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125d3a7-37f4-40e1-811b-16a555a62a0f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.835488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:56.978182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:57.000733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125d3a7-37f4-40e1-811b-16a555a62a0f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:57.046151 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ebbd8-67e6-423f-b599-f90b4a92a837 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:57.070559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45581455-1030-4d38-8d62-1742524095ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:57.071236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45581455-1030-4d38-8d62-1742524095ef None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.071524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45581455-1030-4d38-8d62-1742524095ef None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.071808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45581455-1030-4d38-8d62-1742524095ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.148417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033aee80-8a47-4392-86e0-a8b719cd5cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:57.149465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033aee80-8a47-4392-86e0-a8b719cd5cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.149697 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033aee80-8a47-4392-86e0-a8b719cd5cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.149922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033aee80-8a47-4392-86e0-a8b719cd5cd2 None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.151277 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033aee80-8a47-4392-86e0-a8b719cd5cd2 None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:57.152863 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033aee80-8a47-4392-86e0-a8b719cd5cd2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:57.163226 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 45/88] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:57.164149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033aee80-8a47-4392-86e0-a8b719cd5cd2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:57.182838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45581455-1030-4d38-8d62-1742524095ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:57.184815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45581455-1030-4d38-8d62-1742524095ef None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:57.277664 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:57.302609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033aee80-8a47-4392-86e0-a8b719cd5cd2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:57.352823 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246a7ca-154d-450e-b152-4b95d52bee57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:57.353551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246a7ca-154d-450e-b152-4b95d52bee57 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.353837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246a7ca-154d-450e-b152-4b95d52bee57 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.354115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246a7ca-154d-450e-b152-4b95d52bee57 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.354884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246a7ca-154d-450e-b152-4b95d52bee57 None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:57.355195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246a7ca-154d-450e-b152-4b95d52bee57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:57.356662 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246a7ca-154d-450e-b152-4b95d52bee57 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:57.369383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 46/89] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:20:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:57.370647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246a7ca-154d-450e-b152-4b95d52bee57 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:57.446967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:57.467005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 44/90] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:20:57.469136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45581455-1030-4d38-8d62-1742524095ef None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:57.564444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246a7ca-154d-450e-b152-4b95d52bee57 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:57.619913 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d4155-ebad-4ce7-9012-979f97e9cb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:57.620588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d4155-ebad-4ce7-9012-979f97e9cb00 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.620873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d4155-ebad-4ce7-9012-979f97e9cb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.621174 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d4155-ebad-4ce7-9012-979f97e9cb00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.622026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d4155-ebad-4ce7-9012-979f97e9cb00 None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=d00b803196664866a4fd2eb970b1be0b, user.id=d70de4148c8f4696ac913ae91f5a81cb) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:57.622344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d4155-ebad-4ce7-9012-979f97e9cb00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:57.624512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d4155-ebad-4ce7-9012-979f97e9cb00 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:57.638163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 47/91] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:57 2022] GET /identity/v3/role_assignments?user.id=d70de4148c8f4696ac913ae91f5a81cb&scope.project.id=d00b803196664866a4fd2eb970b1be0b&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:57.639046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d4155-ebad-4ce7-9012-979f97e9cb00 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:57.661504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45581455-1030-4d38-8d62-1742524095ef None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:57.662277 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:57.739568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1bad7-8c60-494d-b975-a849b4e77d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:57.740923 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1bad7-8c60-494d-b975-a849b4e77d3c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.742910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1bad7-8c60-494d-b975-a849b4e77d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.743579 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1bad7-8c60-494d-b975-a849b4e77d3c None admin] PATH_INFO: `/v3/users/d49ddf0ef68c45e4a61eddb790691a24` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.747124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1bad7-8c60-494d-b975-a849b4e77d3c None admin] RBAC: Authorizing `identity:get_user(user_id=d49ddf0ef68c45e4a61eddb790691a24)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:57.749533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1bad7-8c60-494d-b975-a849b4e77d3c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:57.758737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 45/92] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:20:57 2022] GET /identity/v3/users/d49ddf0ef68c45e4a61eddb790691a24 => generated 275 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:57.764531 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1bad7-8c60-494d-b975-a849b4e77d3c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:57.914574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d4155-ebad-4ce7-9012-979f97e9cb00 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:57.938737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:57.998146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34c13e-c936-418c-ad44-321692637f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:57.999486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34c13e-c936-418c-ad44-321692637f00 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.999761 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34c13e-c936-418c-ad44-321692637f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:58.000039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34c13e-c936-418c-ad44-321692637f00 None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:58.002784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34c13e-c936-418c-ad44-321692637f00 None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:58.004263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34c13e-c936-418c-ad44-321692637f00 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:58.006298 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 48/93] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:20:57 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:58.012339 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34c13e-c936-418c-ad44-321692637f00 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:58.036400 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1bad7-8c60-494d-b975-a849b4e77d3c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:58.122917 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71ca86-9245-455f-9397-ba5dd98eb75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:58.123590 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71ca86-9245-455f-9397-ba5dd98eb75f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:58.123925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71ca86-9245-455f-9397-ba5dd98eb75f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:58.124203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71ca86-9245-455f-9397-ba5dd98eb75f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:58.188004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:58.233207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34c13e-c936-418c-ad44-321692637f00 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:58.260171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71ca86-9245-455f-9397-ba5dd98eb75f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:58.261617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71ca86-9245-455f-9397-ba5dd98eb75f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:58.313567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4c3d6-7cfb-4b69-b21c-662b1f20b330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:58.314240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4c3d6-7cfb-4b69-b21c-662b1f20b330 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:58.314531 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4c3d6-7cfb-4b69-b21c-662b1f20b330 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:58.314813 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4c3d6-7cfb-4b69-b21c-662b1f20b330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:58.315653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba4c3d6-7cfb-4b69-b21c-662b1f20b330 None admin] RBAC: Adding query filter params (role.id=5addc8ffe20f47749792ac1394f6235c, scope.project.id=d00b803196664866a4fd2eb970b1be0b, user.id=d49ddf0ef68c45e4a61eddb790691a24) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:58.315968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba4c3d6-7cfb-4b69-b21c-662b1f20b330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:58.317438 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba4c3d6-7cfb-4b69-b21c-662b1f20b330 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:58.335999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 49/94] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:58 2022] GET /identity/v3/role_assignments?user.id=d49ddf0ef68c45e4a61eddb790691a24&scope.project.id=d00b803196664866a4fd2eb970b1be0b&role.id=5addc8ffe20f47749792ac1394f6235c => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:58.361594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:58.414101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 46/95] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:58.422315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4c3d6-7cfb-4b69-b21c-662b1f20b330 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:58.555026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4c3d6-7cfb-4b69-b21c-662b1f20b330 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:58.596817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060de2b-e2c5-4cfa-8f8e-d2d1210f8311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:58.597755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060de2b-e2c5-4cfa-8f8e-d2d1210f8311 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:58.598042 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060de2b-e2c5-4cfa-8f8e-d2d1210f8311 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:58.598322 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060de2b-e2c5-4cfa-8f8e-d2d1210f8311 None admin] PATH_INFO: `/v3/projects/7b2c588cb1484b8caf109e0c625cc831` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:58.606163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060de2b-e2c5-4cfa-8f8e-d2d1210f8311 None admin] RBAC: Authorizing `identity:get_project(project_id=7b2c588cb1484b8caf109e0c625cc831)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:58.607710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060de2b-e2c5-4cfa-8f8e-d2d1210f8311 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:58.610155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 50/96] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:20:58 2022] GET /identity/v3/projects/7b2c588cb1484b8caf109e0c625cc831 => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:58.616252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71ca86-9245-455f-9397-ba5dd98eb75f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:58.758845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71ca86-9245-455f-9397-ba5dd98eb75f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:58.800231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1dbdac-697f-4384-8ec1-8e6fc18c828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:58.800916 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1dbdac-697f-4384-8ec1-8e6fc18c828f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:58.801239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1dbdac-697f-4384-8ec1-8e6fc18c828f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:58.801521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1dbdac-697f-4384-8ec1-8e6fc18c828f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:58.802378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1dbdac-697f-4384-8ec1-8e6fc18c828f None admin] RBAC: Adding query filter params (role.id=2bdabfafa05e443e842fe34d5f03e2c9, scope.project.id=7b2c588cb1484b8caf109e0c625cc831, user.id=d49ddf0ef68c45e4a61eddb790691a24) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:58.802696 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1dbdac-697f-4384-8ec1-8e6fc18c828f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:58.804206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1dbdac-697f-4384-8ec1-8e6fc18c828f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:58.815558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 47/97] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:58 2022] GET /identity/v3/role_assignments?user.id=d49ddf0ef68c45e4a61eddb790691a24&scope.project.id=7b2c588cb1484b8caf109e0c625cc831&role.id=2bdabfafa05e443e842fe34d5f03e2c9 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:59.229037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf4dd4-ca07-47c3-b5f6-2f3040395ba6 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:59.229422 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf4dd4-ca07-47c3-b5f6-2f3040395ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:59.229632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf4dd4-ca07-47c3-b5f6-2f3040395ba6 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:59.231315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 51/98] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:59.236373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28059da1-4746-47ef-b074-46d601a25f62 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:59.236670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28059da1-4746-47ef-b074-46d601a25f62 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:59.236953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28059da1-4746-47ef-b074-46d601a25f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:59.257456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-28059da1-4746-47ef-b074-46d601a25f62 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:59.258663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-28059da1-4746-47ef-b074-46d601a25f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:59.288718 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:59.351948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 48/99] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:59.387434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf4dd4-ca07-47c3-b5f6-2f3040395ba6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:59.397355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:59.505920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf4dd4-ca07-47c3-b5f6-2f3040395ba6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:59.547986 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de77b8-3e89-4b99-84fa-75140419f976 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:59.548326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de77b8-3e89-4b99-84fa-75140419f976 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:59.548569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de77b8-3e89-4b99-84fa-75140419f976 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:59.553667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 49/100] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:20:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:59.556741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4dbc1-2bee-4bd1-a60b-259fda3a5f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:59.557589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4dbc1-2bee-4bd1-a60b-259fda3a5f6d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:59.557920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4dbc1-2bee-4bd1-a60b-259fda3a5f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:59.558209 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4dbc1-2bee-4bd1-a60b-259fda3a5f6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:59.559458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a11e4a-7bf9-4ee8-8e93-96c3c0feb37e None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:59.559756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a11e4a-7bf9-4ee8-8e93-96c3c0feb37e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:59.560040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a11e4a-7bf9-4ee8-8e93-96c3c0feb37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:59.587671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4dbc1-2bee-4bd1-a60b-259fda3a5f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:59.608671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-82a11e4a-7bf9-4ee8-8e93-96c3c0feb37e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:59.610216 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-82a11e4a-7bf9-4ee8-8e93-96c3c0feb37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f5b4dbc1-2bee-4bd1-a60b-259fda3a5f6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:59.615449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:59.616663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:59.617469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:59.617469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:59.617469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:59.617469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:59.617469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:59.617469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:59.617469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:59.617469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:59.617469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:59.617469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:59.617469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:20:59.617469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:20:59.617469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 52/101] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:59.626215 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4dbc1-2bee-4bd1-a60b-259fda3a5f6d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:59.728994 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:59.775685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 50/102] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:59.796065 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4dbc1-2bee-4bd1-a60b-259fda3a5f6d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:59.851316 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a11e4a-7bf9-4ee8-8e93-96c3c0feb37e None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:59.898294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a190bfc-baf6-428b-8199-2a7ee3f4b828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:59.899822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a190bfc-baf6-428b-8199-2a7ee3f4b828 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:59.900508 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a190bfc-baf6-428b-8199-2a7ee3f4b828 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:59.901012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a190bfc-baf6-428b-8199-2a7ee3f4b828 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:59.904181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a190bfc-baf6-428b-8199-2a7ee3f4b828 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:59.904506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a190bfc-baf6-428b-8199-2a7ee3f4b828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:59.906976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a190bfc-baf6-428b-8199-2a7ee3f4b828 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:59.939750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 53/103] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:20:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:59.946106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a190bfc-baf6-428b-8199-2a7ee3f4b828 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.086234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:00.124099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a190bfc-baf6-428b-8199-2a7ee3f4b828 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.134701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a11e4a-7bf9-4ee8-8e93-96c3c0feb37e None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.159569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:00.175261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb387dac-37cb-4698-b8e0-699253137fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.178108 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb387dac-37cb-4698-b8e0-699253137fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.179417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb387dac-37cb-4698-b8e0-699253137fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.182776 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb387dac-37cb-4698-b8e0-699253137fc4 None admin] PATH_INFO: `/v3/users/9aa008bceb70450a92b10af150f8e8a8` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.208976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb387dac-37cb-4698-b8e0-699253137fc4 None admin] RBAC: Authorizing `identity:get_user(user_id=9aa008bceb70450a92b10af150f8e8a8)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.210813 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb387dac-37cb-4698-b8e0-699253137fc4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.213122 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 54/104] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:20:59 2022] GET /identity/v3/users/9aa008bceb70450a92b10af150f8e8a8 => generated 289 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:00.219724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c70e05-0117-4d23-b98e-7aefa525440c None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.219910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd00f35-c737-4376-90c9-65cdefc44296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.219910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd00f35-c737-4376-90c9-65cdefc44296 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.219910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd00f35-c737-4376-90c9-65cdefc44296 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.219910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd00f35-c737-4376-90c9-65cdefc44296 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.220811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c70e05-0117-4d23-b98e-7aefa525440c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.221339 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c70e05-0117-4d23-b98e-7aefa525440c None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.221801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd00f35-c737-4376-90c9-65cdefc44296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.223239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 55/105] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:00.224232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c70e05-0117-4d23-b98e-7aefa525440c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.227490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd00f35-c737-4376-90c9-65cdefc44296 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.232845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 51/106] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:00.232845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fbb54-b342-45d8-b5e5-0ff732b82ebc None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.232845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fbb54-b342-45d8-b5e5-0ff732b82ebc None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.232845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fbb54-b342-45d8-b5e5-0ff732b82ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.258194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-ef1fbb54-b342-45d8-b5e5-0ff732b82ebc None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:00.261314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ef1fbb54-b342-45d8-b5e5-0ff732b82ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:00.405907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 52/107] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:00.409224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fbb54-b342-45d8-b5e5-0ff732b82ebc None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.445354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c70e05-0117-4d23-b98e-7aefa525440c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.565603 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e14f3-1d26-4dff-9aec-604c7e53bc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.568708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e14f3-1d26-4dff-9aec-604c7e53bc67 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.569719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e14f3-1d26-4dff-9aec-604c7e53bc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.572535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e14f3-1d26-4dff-9aec-604c7e53bc67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.691799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:00.735236 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:00.737078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fbb54-b342-45d8-b5e5-0ff732b82ebc None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.786059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e14f3-1d26-4dff-9aec-604c7e53bc67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.787568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e14f3-1d26-4dff-9aec-604c7e53bc67 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.821880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e128b0-5ece-41f2-9c1a-7f4d4741a824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.823988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e128b0-5ece-41f2-9c1a-7f4d4741a824 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.824739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e128b0-5ece-41f2-9c1a-7f4d4741a824 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.828224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e128b0-5ece-41f2-9c1a-7f4d4741a824 None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.829932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e128b0-5ece-41f2-9c1a-7f4d4741a824 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.834943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e128b0-5ece-41f2-9c1a-7f4d4741a824 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.921248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 53/108] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:21:00 2022] POST /identity/v3/projects => generated 302 bytes in 514 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:00.922848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e128b0-5ece-41f2-9c1a-7f4d4741a824 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:01.098178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 56/109] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 874 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:01.104274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e14f3-1d26-4dff-9aec-604c7e53bc67 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:01.141675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e128b0-5ece-41f2-9c1a-7f4d4741a824 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:01.230402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063a1b9-8a14-4502-a14a-cc1d3e64559c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:01.231470 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063a1b9-8a14-4502-a14a-cc1d3e64559c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:01.231762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063a1b9-8a14-4502-a14a-cc1d3e64559c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:01.232045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063a1b9-8a14-4502-a14a-cc1d3e64559c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:01.236939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:01.250526 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063a1b9-8a14-4502-a14a-cc1d3e64559c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-3063a1b9-8a14-4502-a14a-cc1d3e64559c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:01.261981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:01.262903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:01.263673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:01.263673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:01.263673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:01.263673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:01.263673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:01.263673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:01.263673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:01.263673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:01.263673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:01.263673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:01.263673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:01.263673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:01.271657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 54/110] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:01.273627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37170e4-9c74-4a99-adac-ba826a9b8fec None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:01.274071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37170e4-9c74-4a99-adac-ba826a9b8fec None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:01.274646 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37170e4-9c74-4a99-adac-ba826a9b8fec None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:01.276373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 55/111] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:01.277151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37170e4-9c74-4a99-adac-ba826a9b8fec None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:01.289419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:01.308572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e14f3-1d26-4dff-9aec-604c7e53bc67 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:01.354189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea74e4-2ecb-4d19-8b4d-bbd46eafef33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:01.355429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea74e4-2ecb-4d19-8b4d-bbd46eafef33 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:01.355692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea74e4-2ecb-4d19-8b4d-bbd46eafef33 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:01.356072 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea74e4-2ecb-4d19-8b4d-bbd46eafef33 None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:01.357477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea74e4-2ecb-4d19-8b4d-bbd46eafef33 None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:01.360171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea74e4-2ecb-4d19-8b4d-bbd46eafef33 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:01.362002 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 57/112] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:01 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:01.363396 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94e553-d121-47ce-9a91-fae20e41e2cc None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:01.363838 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94e553-d121-47ce-9a91-fae20e41e2cc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:01.364220 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94e553-d121-47ce-9a91-fae20e41e2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:01.385498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-8d94e553-d121-47ce-9a91-fae20e41e2cc None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:01.387051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8d94e553-d121-47ce-9a91-fae20e41e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:01.428142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37170e4-9c74-4a99-adac-ba826a9b8fec None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:01.479131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 58/113] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:01.480028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94e553-d121-47ce-9a91-fae20e41e2cc None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:01.494968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcdc9b-9118-4391-981d-cef8fa9e653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:01.495700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcdc9b-9118-4391-981d-cef8fa9e653e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:01.495994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcdc9b-9118-4391-981d-cef8fa9e653e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:01.496270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcdc9b-9118-4391-981d-cef8fa9e653e None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:01.497182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcdc9b-9118-4391-981d-cef8fa9e653e None admin] RBAC: Adding query filter params (name=member) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:01.497506 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcdc9b-9118-4391-981d-cef8fa9e653e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:01.504060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcdc9b-9118-4391-981d-cef8fa9e653e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:01.519583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 56/114] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:01.519583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f79f3-771b-4169-bfa1-4f729a7fd846 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:01.519583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f79f3-771b-4169-bfa1-4f729a7fd846 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:01.519583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f79f3-771b-4169-bfa1-4f729a7fd846 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:01.519583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 57/115] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:01.520220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f79f3-771b-4169-bfa1-4f729a7fd846 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:01.604446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:01.619471 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:01.645584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94e553-d121-47ce-9a91-fae20e41e2cc None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:01.667973 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f79f3-771b-4169-bfa1-4f729a7fd846 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:01.688643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007aa45-a9a4-458e-86e5-b9a1e0281065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:01.689750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007aa45-a9a4-458e-86e5-b9a1e0281065 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:01.690247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007aa45-a9a4-458e-86e5-b9a1e0281065 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:01.690748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007aa45-a9a4-458e-86e5-b9a1e0281065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:01.691845 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007aa45-a9a4-458e-86e5-b9a1e0281065 None admin] RBAC: Adding query filter params (role.id=6a4f5f6d39ed47f4a5036b99c7386949, scope.project.id=d00b803196664866a4fd2eb970b1be0b, user.id=9aa008bceb70450a92b10af150f8e8a8) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:01.692381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007aa45-a9a4-458e-86e5-b9a1e0281065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:01.694218 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007aa45-a9a4-458e-86e5-b9a1e0281065 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:01.701515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 59/116] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:01 2022] GET /identity/v3/role_assignments?user.id=9aa008bceb70450a92b10af150f8e8a8&scope.project.id=d00b803196664866a4fd2eb970b1be0b&role.id=6a4f5f6d39ed47f4a5036b99c7386949 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:01.702843 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007aa45-a9a4-458e-86e5-b9a1e0281065 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:01.727684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf9d63-6084-4356-986e-88ebc08c534f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:01.728864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf9d63-6084-4356-986e-88ebc08c534f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:01.730207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf9d63-6084-4356-986e-88ebc08c534f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:01.730900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf9d63-6084-4356-986e-88ebc08c534f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:01.757750 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf9d63-6084-4356-986e-88ebc08c534f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-5fbf9d63-6084-4356-986e-88ebc08c534f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:01.768977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:01.769926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:01.770693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:01.770693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:01.770693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:01.770693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:01.770693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:01.770693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:01.770693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:01.770693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:01.770693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:01.770693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:01.770693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:01.770693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:01.781698 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 58/117] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:01.784458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579390d-a158-4053-b48d-8b3086347707 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:01.790464 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579390d-a158-4053-b48d-8b3086347707 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:01.791112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579390d-a158-4053-b48d-8b3086347707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:01.816262 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e579390d-a158-4053-b48d-8b3086347707 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:01.818037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e579390d-a158-4053-b48d-8b3086347707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:01.908157 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007aa45-a9a4-458e-86e5-b9a1e0281065 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:01.971275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 59/118] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:01.973505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579390d-a158-4053-b48d-8b3086347707 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:01.976460 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e4aa0-0060-4846-9a14-12762766cb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:01.977491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e4aa0-0060-4846-9a14-12762766cb79 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:01.977945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e4aa0-0060-4846-9a14-12762766cb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:01.978402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e4aa0-0060-4846-9a14-12762766cb79 None admin] PATH_INFO: `/v3/users/d49ddf0ef68c45e4a61eddb790691a24` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:01.980634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e4aa0-0060-4846-9a14-12762766cb79 None admin] RBAC: Authorizing `identity:get_user(user_id=d49ddf0ef68c45e4a61eddb790691a24)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:01.982818 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e4aa0-0060-4846-9a14-12762766cb79 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:01.985498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 60/119] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:01 2022] GET /identity/v3/users/d49ddf0ef68c45e4a61eddb790691a24 => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:01.993248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e4aa0-0060-4846-9a14-12762766cb79 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:02.033636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:02.058880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:02.115855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579390d-a158-4053-b48d-8b3086347707 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:02.137211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e4aa0-0060-4846-9a14-12762766cb79 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:02.175767 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f6adf-e36d-49ab-80b7-968e014b1ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:02.176358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f6adf-e36d-49ab-80b7-968e014b1ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:02.176647 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f6adf-e36d-49ab-80b7-968e014b1ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:02.176925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f6adf-e36d-49ab-80b7-968e014b1ea2 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:02.178201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f6adf-e36d-49ab-80b7-968e014b1ea2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:02.178512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f6adf-e36d-49ab-80b7-968e014b1ea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:02.180238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f6adf-e36d-49ab-80b7-968e014b1ea2 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:02.189823 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e855128-4edc-4718-990d-7443467702fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:02.191168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e855128-4edc-4718-990d-7443467702fd None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:02.191617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 60/120] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:02.193179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f6adf-e36d-49ab-80b7-968e014b1ea2 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:02.197798 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e855128-4edc-4718-990d-7443467702fd None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:02.198354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e855128-4edc-4718-990d-7443467702fd None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:02.201015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e855128-4edc-4718-990d-7443467702fd None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:02.203931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e855128-4edc-4718-990d-7443467702fd None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:02.205953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 61/121] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:01 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:02.207335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cd9e5-0721-4805-9072-cbea97b37876 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:02.207698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cd9e5-0721-4805-9072-cbea97b37876 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:02.208126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cd9e5-0721-4805-9072-cbea97b37876 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:02.209552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 62/122] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:02.210347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cd9e5-0721-4805-9072-cbea97b37876 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:02.325307 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:02.333856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f6adf-e36d-49ab-80b7-968e014b1ea2 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:02.347291 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:02.367722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cd9e5-0721-4805-9072-cbea97b37876 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:02.374843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76fbab-57c5-4e71-8c21-5b13950ab65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:02.375665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76fbab-57c5-4e71-8c21-5b13950ab65a None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:02.375954 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76fbab-57c5-4e71-8c21-5b13950ab65a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:02.376235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76fbab-57c5-4e71-8c21-5b13950ab65a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:02.388125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76fbab-57c5-4e71-8c21-5b13950ab65a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-0b76fbab-57c5-4e71-8c21-5b13950ab65a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:02.396838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:02.397784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:02.398538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:02.398538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:02.398538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:02.398538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:02.398538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:02.398538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:02.398538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:02.398538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:02.398538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:02.398538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:02.398538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:02.398538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:02.398538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 61/123] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:21:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:02.399166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76fbab-57c5-4e71-8c21-5b13950ab65a None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:02.424147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80168e-1a5d-4513-b37f-53fc6d1f2262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:02.426237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80168e-1a5d-4513-b37f-53fc6d1f2262 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:02.426710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80168e-1a5d-4513-b37f-53fc6d1f2262 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:02.427163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80168e-1a5d-4513-b37f-53fc6d1f2262 None admin] PATH_INFO: `/v3/users/d70de4148c8f4696ac913ae91f5a81cb` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:02.429283 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80168e-1a5d-4513-b37f-53fc6d1f2262 None admin] RBAC: Authorizing `identity:get_user(user_id=d70de4148c8f4696ac913ae91f5a81cb)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:02.430966 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80168e-1a5d-4513-b37f-53fc6d1f2262 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:02.433150 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 63/124] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:02 2022] GET /identity/v3/users/d70de4148c8f4696ac913ae91f5a81cb => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:02.434568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a88472-033e-4156-8a40-7b3532bad370 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:02.435059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a88472-033e-4156-8a40-7b3532bad370 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:02.435559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a88472-033e-4156-8a40-7b3532bad370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:02.456900 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-99a88472-033e-4156-8a40-7b3532bad370 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:02.459683 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-99a88472-033e-4156-8a40-7b3532bad370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:02.545964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76fbab-57c5-4e71-8c21-5b13950ab65a None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:02.567918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 64/125] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:02.569582 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a88472-033e-4156-8a40-7b3532bad370 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:02.595461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b154be9-956e-4d87-a20d-4636738d3a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:02.596312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b154be9-956e-4d87-a20d-4636738d3a5a None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:02.596658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b154be9-956e-4d87-a20d-4636738d3a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:02.596972 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b154be9-956e-4d87-a20d-4636738d3a5a None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b/users/d49ddf0ef68c45e4a61eddb790691a24/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:02.599542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b154be9-956e-4d87-a20d-4636738d3a5a None admin] RBAC: Authorizing `identity:create_grant(project_id=d00b803196664866a4fd2eb970b1be0b, user_id=d49ddf0ef68c45e4a61eddb790691a24, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:02.601959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b154be9-956e-4d87-a20d-4636738d3a5a None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:02.627279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 62/126] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:21:02 2022] PUT /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b/users/d49ddf0ef68c45e4a61eddb790691a24/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:02.628005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b154be9-956e-4d87-a20d-4636738d3a5a None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:02.668516 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:02.690801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:02.714398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a88472-033e-4156-8a40-7b3532bad370 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:02.754005 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0e167-8477-4301-a3f7-06304ed9ee16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:02.754937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0e167-8477-4301-a3f7-06304ed9ee16 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:02.755444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0e167-8477-4301-a3f7-06304ed9ee16 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:02.755956 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0e167-8477-4301-a3f7-06304ed9ee16 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:02.756952 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0e167-8477-4301-a3f7-06304ed9ee16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:02.757513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0e167-8477-4301-a3f7-06304ed9ee16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:02.759182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0e167-8477-4301-a3f7-06304ed9ee16 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:02.770241 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b154be9-956e-4d87-a20d-4636738d3a5a None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:02.770741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 65/127] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:02.771821 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0e167-8477-4301-a3f7-06304ed9ee16 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:02.860682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c00d2-2f3b-4231-83fe-37ba5f3ff47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:02.861599 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c00d2-2f3b-4231-83fe-37ba5f3ff47c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:02.861892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c00d2-2f3b-4231-83fe-37ba5f3ff47c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:02.862176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c00d2-2f3b-4231-83fe-37ba5f3ff47c None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:02.864346 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c00d2-2f3b-4231-83fe-37ba5f3ff47c None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:02.865882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c00d2-2f3b-4231-83fe-37ba5f3ff47c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:02.873208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 63/128] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:02 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:02.874898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c00d2-2f3b-4231-83fe-37ba5f3ff47c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:02.967658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0e167-8477-4301-a3f7-06304ed9ee16 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.006914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26a780-eca7-4e95-8dc6-355bcb6aac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.008082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26a780-eca7-4e95-8dc6-355bcb6aac54 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.008609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26a780-eca7-4e95-8dc6-355bcb6aac54 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.009210 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26a780-eca7-4e95-8dc6-355bcb6aac54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.020798 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26a780-eca7-4e95-8dc6-355bcb6aac54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-8f26a780-eca7-4e95-8dc6-355bcb6aac54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:03.028797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:03.029846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:03.030653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:03.030653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:03.030653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:03.030653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:03.030653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:03.030653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:03.030653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:03.030653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:03.030653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:03.030653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:03.030653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:03.030653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:03.032187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:03.034279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 66/129] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:03.035434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26a780-eca7-4e95-8dc6-355bcb6aac54 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.041039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c00d2-2f3b-4231-83fe-37ba5f3ff47c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.042111 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:03.082561 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886473b-b488-49ce-99c2-f533730fccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.083848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886473b-b488-49ce-99c2-f533730fccaa None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.084374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886473b-b488-49ce-99c2-f533730fccaa None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.084854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886473b-b488-49ce-99c2-f533730fccaa None admin] PATH_INFO: `/v3/users/d49ddf0ef68c45e4a61eddb790691a24` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.086444 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886473b-b488-49ce-99c2-f533730fccaa None admin] RBAC: Authorizing `identity:get_user(user_id=d49ddf0ef68c45e4a61eddb790691a24)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:03.088254 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886473b-b488-49ce-99c2-f533730fccaa None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:03.090584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 64/130] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:02 2022] GET /identity/v3/users/d49ddf0ef68c45e4a61eddb790691a24 => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:03.091632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886473b-b488-49ce-99c2-f533730fccaa None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.146069 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26a780-eca7-4e95-8dc6-355bcb6aac54 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.187890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977db00c-182a-42d8-aaef-021b39090c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.189034 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977db00c-182a-42d8-aaef-021b39090c36 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.189499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977db00c-182a-42d8-aaef-021b39090c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.189922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977db00c-182a-42d8-aaef-021b39090c36 None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b/users/d70de4148c8f4696ac913ae91f5a81cb/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.193579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977db00c-182a-42d8-aaef-021b39090c36 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00b803196664866a4fd2eb970b1be0b, user_id=d70de4148c8f4696ac913ae91f5a81cb, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:03.195256 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977db00c-182a-42d8-aaef-021b39090c36 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:03.218879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 67/131] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:21:03 2022] PUT /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b/users/d70de4148c8f4696ac913ae91f5a81cb/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:03.220016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977db00c-182a-42d8-aaef-021b39090c36 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.278265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886473b-b488-49ce-99c2-f533730fccaa None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.301503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:03.361166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ba098-c458-4da9-85e8-6883980555c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.361878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ba098-c458-4da9-85e8-6883980555c4 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.362169 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ba098-c458-4da9-85e8-6883980555c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.362453 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ba098-c458-4da9-85e8-6883980555c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.363310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ba098-c458-4da9-85e8-6883980555c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:03.364108 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ba098-c458-4da9-85e8-6883980555c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:03.364989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ba098-c458-4da9-85e8-6883980555c4 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:03.372198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977db00c-182a-42d8-aaef-021b39090c36 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.381635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 65/132] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:03.387376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ba098-c458-4da9-85e8-6883980555c4 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.452175 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:03.487568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad292c-763e-4668-bcb3-2383465145a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.488382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad292c-763e-4668-bcb3-2383465145a9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.488670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad292c-763e-4668-bcb3-2383465145a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.488951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad292c-763e-4668-bcb3-2383465145a9 None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.493968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad292c-763e-4668-bcb3-2383465145a9 None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:03.495470 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad292c-763e-4668-bcb3-2383465145a9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:03.497739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 68/133] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:03 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:03.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad292c-763e-4668-bcb3-2383465145a9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.624609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ba098-c458-4da9-85e8-6883980555c4 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.679037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad292c-763e-4668-bcb3-2383465145a9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.710286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7d7bc-4508-4e80-ac41-a81e24e81553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.711227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7d7bc-4508-4e80-ac41-a81e24e81553 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.711514 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7d7bc-4508-4e80-ac41-a81e24e81553 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.711789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7d7bc-4508-4e80-ac41-a81e24e81553 None admin] PATH_INFO: `/v3/users/d49ddf0ef68c45e4a61eddb790691a24` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.713505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7d7bc-4508-4e80-ac41-a81e24e81553 None admin] RBAC: Authorizing `identity:get_user(user_id=d49ddf0ef68c45e4a61eddb790691a24)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:03.715000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7d7bc-4508-4e80-ac41-a81e24e81553 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:03.716670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 66/134] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:03 2022] GET /identity/v3/users/d49ddf0ef68c45e4a61eddb790691a24 => generated 275 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:03.723323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7d7bc-4508-4e80-ac41-a81e24e81553 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.736322 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:03.742988 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82d18b-cc8b-40fc-b83c-15d3063eb028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.744375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82d18b-cc8b-40fc-b83c-15d3063eb028 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.744992 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82d18b-cc8b-40fc-b83c-15d3063eb028 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.745489 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82d18b-cc8b-40fc-b83c-15d3063eb028 None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b/users/d49ddf0ef68c45e4a61eddb790691a24/roles/5addc8ffe20f47749792ac1394f6235c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.749227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82d18b-cc8b-40fc-b83c-15d3063eb028 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00b803196664866a4fd2eb970b1be0b, user_id=d49ddf0ef68c45e4a61eddb790691a24, role_id=5addc8ffe20f47749792ac1394f6235c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:03.751624 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82d18b-cc8b-40fc-b83c-15d3063eb028 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:03.778953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 69/135] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:21:03 2022] PUT /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b/users/d49ddf0ef68c45e4a61eddb790691a24/roles/5addc8ffe20f47749792ac1394f6235c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:03.946668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:03.956143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7d7bc-4508-4e80-ac41-a81e24e81553 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:04.001474 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bd509-9f98-4614-b791-ffa2cdeeb4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:04.002276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bd509-9f98-4614-b791-ffa2cdeeb4ce None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:04.002559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bd509-9f98-4614-b791-ffa2cdeeb4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:04.002835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bd509-9f98-4614-b791-ffa2cdeeb4ce None admin] PATH_INFO: `/v3/projects/7b2c588cb1484b8caf109e0c625cc831` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:04.004086 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bd509-9f98-4614-b791-ffa2cdeeb4ce None admin] RBAC: Authorizing `identity:get_project(project_id=7b2c588cb1484b8caf109e0c625cc831)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:04.005614 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bd509-9f98-4614-b791-ffa2cdeeb4ce None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:04.007554 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 67/136] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:03 2022] GET /identity/v3/projects/7b2c588cb1484b8caf109e0c625cc831 => generated 312 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:04.013341 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82d18b-cc8b-40fc-b83c-15d3063eb028 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:04.107062 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffdd0a-4b6c-4611-be56-aa5837e111d0 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:04.107708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffdd0a-4b6c-4611-be56-aa5837e111d0 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:04.107708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffdd0a-4b6c-4611-be56-aa5837e111d0 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:04.110327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 68/137] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:04.116104 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b9e4e-b8cc-4d37-9e12-92dc4a3cebeb None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:04.116408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b9e4e-b8cc-4d37-9e12-92dc4a3cebeb None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:04.116663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b9e4e-b8cc-4d37-9e12-92dc4a3cebeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:04.146664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-5d5b9e4e-b8cc-4d37-9e12-92dc4a3cebeb None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:04.148785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5d5b9e4e-b8cc-4d37-9e12-92dc4a3cebeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:04.230172 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82d18b-cc8b-40fc-b83c-15d3063eb028 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:04.319048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ccee0-5750-4156-93b3-2aa59e53cdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:04.320816 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ccee0-5750-4156-93b3-2aa59e53cdc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:04.321323 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ccee0-5750-4156-93b3-2aa59e53cdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:04.321777 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ccee0-5750-4156-93b3-2aa59e53cdc9 None admin] PATH_INFO: `/v3/projects/7b2c588cb1484b8caf109e0c625cc831/users/d49ddf0ef68c45e4a61eddb790691a24/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:04.325206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ccee0-5750-4156-93b3-2aa59e53cdc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b2c588cb1484b8caf109e0c625cc831, user_id=d49ddf0ef68c45e4a61eddb790691a24, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:04.327322 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 69/138] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:04.328705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ccee0-5750-4156-93b3-2aa59e53cdc9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:04.355886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b9e4e-b8cc-4d37-9e12-92dc4a3cebeb None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:04.356865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 70/139] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:21:04 2022] PUT /identity/v3/projects/7b2c588cb1484b8caf109e0c625cc831/users/d49ddf0ef68c45e4a61eddb790691a24/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:04.462317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:04.473230 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b9e4e-b8cc-4d37-9e12-92dc4a3cebeb None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:04.511756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c55d47-c927-4d67-8b71-21a58715311c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:04.512555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c55d47-c927-4d67-8b71-21a58715311c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:04.512842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c55d47-c927-4d67-8b71-21a58715311c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:04.513134 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c55d47-c927-4d67-8b71-21a58715311c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:04.514888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c55d47-c927-4d67-8b71-21a58715311c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:04.516810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c55d47-c927-4d67-8b71-21a58715311c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:04.518786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 70/140] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:04.525019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ccee0-5750-4156-93b3-2aa59e53cdc9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:04.598142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:04.653970 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ccee0-5750-4156-93b3-2aa59e53cdc9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:04.691779 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00e6d6-68c4-4056-928a-6338a007ee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:04.692793 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00e6d6-68c4-4056-928a-6338a007ee8a None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:04.693310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00e6d6-68c4-4056-928a-6338a007ee8a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:04.693831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00e6d6-68c4-4056-928a-6338a007ee8a None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:04.694816 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00e6d6-68c4-4056-928a-6338a007ee8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:04.696565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00e6d6-68c4-4056-928a-6338a007ee8a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:04.721386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 71/141] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:21:04 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:04.823255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8442c-b67a-4b62-88d9-12ea8151e26d None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:04.823828 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8442c-b67a-4b62-88d9-12ea8151e26d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:04.824279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8442c-b67a-4b62-88d9-12ea8151e26d None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:04.826213 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 71/142] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:04.833020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886e0e-1da0-47cf-98b5-01adda0dd060 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:04.833683 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886e0e-1da0-47cf-98b5-01adda0dd060 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:04.834207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886e0e-1da0-47cf-98b5-01adda0dd060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:04.861458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ce886e0e-1da0-47cf-98b5-01adda0dd060 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:04.862963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ce886e0e-1da0-47cf-98b5-01adda0dd060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:04.951476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 72/143] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:04.975960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8442c-b67a-4b62-88d9-12ea8151e26d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:05.106271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8442c-b67a-4b62-88d9-12ea8151e26d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:05.154369 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011a3ce-a1b1-4910-a5bd-8a82fd1c55ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:05.156020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011a3ce-a1b1-4910-a5bd-8a82fd1c55ba None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:05.156504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011a3ce-a1b1-4910-a5bd-8a82fd1c55ba None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:05.156787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011a3ce-a1b1-4910-a5bd-8a82fd1c55ba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:05.168189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011a3ce-a1b1-4910-a5bd-8a82fd1c55ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-2011a3ce-a1b1-4910-a5bd-8a82fd1c55ba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:05.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:05.184367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:05.185150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:05.185150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:05.185150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:05.185150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:05.185150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:05.185150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:05.185150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:05.185150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:05.185150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:05.185150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:05.185150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:05.185150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:05.185150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 72/144] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:05.190752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce886e0e-1da0-47cf-98b5-01adda0dd060 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:05.251783 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:05.333980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce886e0e-1da0-47cf-98b5-01adda0dd060 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:05.377499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe6b73-dd7d-412e-a371-480ddd56a959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:05.378539 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe6b73-dd7d-412e-a371-480ddd56a959 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:05.379056 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe6b73-dd7d-412e-a371-480ddd56a959 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:05.379530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe6b73-dd7d-412e-a371-480ddd56a959 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:05.380481 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe6b73-dd7d-412e-a371-480ddd56a959 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:05.380986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe6b73-dd7d-412e-a371-480ddd56a959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:05.382630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe6b73-dd7d-412e-a371-480ddd56a959 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:05.392479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 73/145] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:05.398528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011a3ce-a1b1-4910-a5bd-8a82fd1c55ba None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:05.437549 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:05.557251 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011a3ce-a1b1-4910-a5bd-8a82fd1c55ba None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:05.599535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a0759-03c8-411f-922a-e74ae085b040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:05.600488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a0759-03c8-411f-922a-e74ae085b040 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:05.600770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a0759-03c8-411f-922a-e74ae085b040 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:05.601108 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a0759-03c8-411f-922a-e74ae085b040 None admin] PATH_INFO: `/v3/users/9aa008bceb70450a92b10af150f8e8a8` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:05.602341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a0759-03c8-411f-922a-e74ae085b040 None admin] RBAC: Authorizing `identity:get_user(user_id=9aa008bceb70450a92b10af150f8e8a8)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:05.603787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a0759-03c8-411f-922a-e74ae085b040 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:05.605502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 73/146] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:05 2022] GET /identity/v3/users/9aa008bceb70450a92b10af150f8e8a8 => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:05.618250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe6b73-dd7d-412e-a371-480ddd56a959 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:05.777956 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:05.781711 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe6b73-dd7d-412e-a371-480ddd56a959 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:05.825453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10f84c-6cde-405c-b81b-dc1621e0206a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:05.826348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10f84c-6cde-405c-b81b-dc1621e0206a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:05.827652 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10f84c-6cde-405c-b81b-dc1621e0206a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:05.828205 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10f84c-6cde-405c-b81b-dc1621e0206a None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:05.829573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10f84c-6cde-405c-b81b-dc1621e0206a None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:05.831058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10f84c-6cde-405c-b81b-dc1621e0206a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:05.833254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 74/147] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:05 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:05.838941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a0759-03c8-411f-922a-e74ae085b040 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:05.969377 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:05.986448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a0759-03c8-411f-922a-e74ae085b040 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:06.033850 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238af617-90ce-4251-b686-4f858ecf81d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:06.034696 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238af617-90ce-4251-b686-4f858ecf81d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.034995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238af617-90ce-4251-b686-4f858ecf81d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.035268 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238af617-90ce-4251-b686-4f858ecf81d9 None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b/users/9aa008bceb70450a92b10af150f8e8a8/roles/6a4f5f6d39ed47f4a5036b99c7386949` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.042255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238af617-90ce-4251-b686-4f858ecf81d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00b803196664866a4fd2eb970b1be0b, user_id=9aa008bceb70450a92b10af150f8e8a8, role_id=6a4f5f6d39ed47f4a5036b99c7386949)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:06.043793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238af617-90ce-4251-b686-4f858ecf81d9 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:06.067363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 74/148] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:21:05 2022] PUT /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b/users/9aa008bceb70450a92b10af150f8e8a8/roles/6a4f5f6d39ed47f4a5036b99c7386949 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:06.445618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833b2ca-5bae-429f-8559-51e543dab7b7 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.445618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833b2ca-5bae-429f-8559-51e543dab7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.445618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833b2ca-5bae-429f-8559-51e543dab7b7 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.446853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 75/149] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:06.451987 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f05ac-82f0-491a-92a3-b2d3be7cf8b5 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.453317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f05ac-82f0-491a-92a3-b2d3be7cf8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.453908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f05ac-82f0-491a-92a3-b2d3be7cf8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.475217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-cc8f05ac-82f0-491a-92a3-b2d3be7cf8b5 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:06.476728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-cc8f05ac-82f0-491a-92a3-b2d3be7cf8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:06.570224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 75/150] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:06.601856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833b2ca-5bae-429f-8559-51e543dab7b7 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:06.736455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833b2ca-5bae-429f-8559-51e543dab7b7 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:06.774358 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80dc51-6254-4c8e-adfe-cf0ce407a3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:06.775157 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80dc51-6254-4c8e-adfe-cf0ce407a3ba None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.775436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80dc51-6254-4c8e-adfe-cf0ce407a3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.775712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80dc51-6254-4c8e-adfe-cf0ce407a3ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.786865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80dc51-6254-4c8e-adfe-cf0ce407a3ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-6d80dc51-6254-4c8e-adfe-cf0ce407a3ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:06.794701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:06.796251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:06.797403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:06.797403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:06.797403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:06.797403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:06.797403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:06.797403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:06.797403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:06.797403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:06.797403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:06.797403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:06.797403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:06.797403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:06.797403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 76/151] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:06.802461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f05ac-82f0-491a-92a3-b2d3be7cf8b5 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:06.945761 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:06.977681 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f05ac-82f0-491a-92a3-b2d3be7cf8b5 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:07.028702 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe8edf-5f21-4239-b189-7123bffeaf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:07.029361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe8edf-5f21-4239-b189-7123bffeaf2d None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.030223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe8edf-5f21-4239-b189-7123bffeaf2d None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.030507 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe8edf-5f21-4239-b189-7123bffeaf2d None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.031306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe8edf-5f21-4239-b189-7123bffeaf2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:07.031619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe8edf-5f21-4239-b189-7123bffeaf2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:07.033116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe8edf-5f21-4239-b189-7123bffeaf2d None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:07.047781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 76/152] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:07.054196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80dc51-6254-4c8e-adfe-cf0ce407a3ba None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:07.107917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:07.190390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80dc51-6254-4c8e-adfe-cf0ce407a3ba None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:07.229359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92992aa7-8e55-4d25-96a4-4225bd01ffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:07.230247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92992aa7-8e55-4d25-96a4-4225bd01ffe4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.230530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92992aa7-8e55-4d25-96a4-4225bd01ffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.230802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92992aa7-8e55-4d25-96a4-4225bd01ffe4 None admin] PATH_INFO: `/v3/users/d49ddf0ef68c45e4a61eddb790691a24` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.232011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92992aa7-8e55-4d25-96a4-4225bd01ffe4 None admin] RBAC: Authorizing `identity:get_user(user_id=d49ddf0ef68c45e4a61eddb790691a24)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:07.234021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92992aa7-8e55-4d25-96a4-4225bd01ffe4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:07.236081 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 77/153] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:07 2022] GET /identity/v3/users/d49ddf0ef68c45e4a61eddb790691a24 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:07.241516 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe8edf-5f21-4239-b189-7123bffeaf2d None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:07.357045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe8edf-5f21-4239-b189-7123bffeaf2d None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:07.393436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede19d33-a19a-4edb-b8a3-b7e19677d101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:07.394480 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede19d33-a19a-4edb-b8a3-b7e19677d101 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.394480 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede19d33-a19a-4edb-b8a3-b7e19677d101 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.395721 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede19d33-a19a-4edb-b8a3-b7e19677d101 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.407228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:07.477403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede19d33-a19a-4edb-b8a3-b7e19677d101 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:07.478974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede19d33-a19a-4edb-b8a3-b7e19677d101 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:07.588548 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 77/154] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:07.594204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92992aa7-8e55-4d25-96a4-4225bd01ffe4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:07.693586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:07.699107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92992aa7-8e55-4d25-96a4-4225bd01ffe4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:07.735117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125925f-8f5f-4ef5-a255-211114894f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:07.736030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125925f-8f5f-4ef5-a255-211114894f54 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.736312 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125925f-8f5f-4ef5-a255-211114894f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.736587 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125925f-8f5f-4ef5-a255-211114894f54 None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.738002 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125925f-8f5f-4ef5-a255-211114894f54 None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:07.739516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125925f-8f5f-4ef5-a255-211114894f54 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:07.741791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 78/155] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:07 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:07.748202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede19d33-a19a-4edb-b8a3-b7e19677d101 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:07.799735 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:07.866220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede19d33-a19a-4edb-b8a3-b7e19677d101 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:07.880250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976da193-a446-4800-a37f-b77cc081c478 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.880549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976da193-a446-4800-a37f-b77cc081c478 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.880830 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976da193-a446-4800-a37f-b77cc081c478 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.882369 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 79/156] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:07.890884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2ce14-297c-4cd0-9437-4b55b37f5ea2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.891174 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2ce14-297c-4cd0-9437-4b55b37f5ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.891450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2ce14-297c-4cd0-9437-4b55b37f5ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.915078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bb90a-b46a-4e29-a1a6-8edf1bf1ff90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:07.916593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bb90a-b46a-4e29-a1a6-8edf1bf1ff90 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.916879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bb90a-b46a-4e29-a1a6-8edf1bf1ff90 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.917168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bb90a-b46a-4e29-a1a6-8edf1bf1ff90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.918022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bb90a-b46a-4e29-a1a6-8edf1bf1ff90 None admin] RBAC: Adding query filter params (role.id=2bdabfafa05e443e842fe34d5f03e2c9, scope.project.id=d00b803196664866a4fd2eb970b1be0b, user.id=d49ddf0ef68c45e4a61eddb790691a24) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:07.918330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bb90a-b46a-4e29-a1a6-8edf1bf1ff90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:07.919786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bb90a-b46a-4e29-a1a6-8edf1bf1ff90 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:07.927032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 78/157] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:07 2022] GET /identity/v3/role_assignments?user.id=d49ddf0ef68c45e4a61eddb790691a24&scope.project.id=d00b803196664866a4fd2eb970b1be0b&role.id=2bdabfafa05e443e842fe34d5f03e2c9 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:07.942394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6eb2ce14-297c-4cd0-9437-4b55b37f5ea2 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:07.943735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6eb2ce14-297c-4cd0-9437-4b55b37f5ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:08.078750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 80/158] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:08.121075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bb90a-b46a-4e29-a1a6-8edf1bf1ff90 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:08.207502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49c46a-b0c0-4911-a933-ae0e9aa7817f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.208385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49c46a-b0c0-4911-a933-ae0e9aa7817f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.209191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49c46a-b0c0-4911-a933-ae0e9aa7817f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.211189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 81/159] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:08.217838 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770368e-983c-4a46-b1c6-3d3edd767cb4 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.218435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770368e-983c-4a46-b1c6-3d3edd767cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.218957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770368e-983c-4a46-b1c6-3d3edd767cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.255890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bb90a-b46a-4e29-a1a6-8edf1bf1ff90 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:08.257241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6770368e-983c-4a46-b1c6-3d3edd767cb4 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:08.258696 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6770368e-983c-4a46-b1c6-3d3edd767cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:08.264460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:08.297025 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79b997-b0df-45b7-afb8-fcac350817c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:08.297978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79b997-b0df-45b7-afb8-fcac350817c3 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.298259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79b997-b0df-45b7-afb8-fcac350817c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.298534 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79b997-b0df-45b7-afb8-fcac350817c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.310568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79b997-b0df-45b7-afb8-fcac350817c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-1c79b997-b0df-45b7-afb8-fcac350817c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:08.319894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:08.320969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:08.321991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:08.321991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:08.321991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:08.321991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:08.321991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:08.321991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:08.321991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:08.321991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:08.321991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:08.321991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:08.321991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:08.321991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:08.324863 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 79/160] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:21:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:08.331493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79b997-b0df-45b7-afb8-fcac350817c3 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:08.468014 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79b997-b0df-45b7-afb8-fcac350817c3 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:08.470863 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 82/161] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:08.504751 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051696f-771b-43b0-8677-1fc9d6410f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:08.505477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051696f-771b-43b0-8677-1fc9d6410f9e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.505760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051696f-771b-43b0-8677-1fc9d6410f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.506053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051696f-771b-43b0-8677-1fc9d6410f9e None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.506808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051696f-771b-43b0-8677-1fc9d6410f9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:08.507114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051696f-771b-43b0-8677-1fc9d6410f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:08.508505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051696f-771b-43b0-8677-1fc9d6410f9e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:08.511637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770368e-983c-4a46-b1c6-3d3edd767cb4 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:08.518071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 80/162] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:08.524485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051696f-771b-43b0-8677-1fc9d6410f9e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:08.557480 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:08.582067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:08.641784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051696f-771b-43b0-8677-1fc9d6410f9e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:08.685635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee30354-d77a-4709-b415-dbe11a3beb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:08.687385 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee30354-d77a-4709-b415-dbe11a3beb1e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.687740 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee30354-d77a-4709-b415-dbe11a3beb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.688084 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee30354-d77a-4709-b415-dbe11a3beb1e None admin] PATH_INFO: `/v3/users/d49ddf0ef68c45e4a61eddb790691a24` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.689744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee30354-d77a-4709-b415-dbe11a3beb1e None admin] RBAC: Authorizing `identity:get_user(user_id=d49ddf0ef68c45e4a61eddb790691a24)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:08.692617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee30354-d77a-4709-b415-dbe11a3beb1e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:08.694470 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 81/163] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:08 2022] GET /identity/v3/users/d49ddf0ef68c45e4a61eddb790691a24 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:08.705195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee30354-d77a-4709-b415-dbe11a3beb1e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:08.748450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770368e-983c-4a46-b1c6-3d3edd767cb4 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:08.809823 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5a9c4-9a14-4dcc-a1c0-1a75496bde9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:08.811012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5a9c4-9a14-4dcc-a1c0-1a75496bde9a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.811482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5a9c4-9a14-4dcc-a1c0-1a75496bde9a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.811935 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5a9c4-9a14-4dcc-a1c0-1a75496bde9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.829588 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:08.830435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5a9c4-9a14-4dcc-a1c0-1a75496bde9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:08.837394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee30354-d77a-4709-b415-dbe11a3beb1e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-c1d5a9c4-9a14-4dcc-a1c0-1a75496bde9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:08.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:08.839623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:08.840410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:08.840410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:08.840410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:08.840410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:08.840410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:08.840410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:08.840410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:08.840410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:08.840410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:08.840410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:08.840410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:08.840410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:08.842882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 83/164] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:08.848924 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5a9c4-9a14-4dcc-a1c0-1a75496bde9a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:08.903348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36813c1-ffaa-412d-b909-3993be1d78d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:08.904018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36813c1-ffaa-412d-b909-3993be1d78d2 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.904303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36813c1-ffaa-412d-b909-3993be1d78d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.904733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36813c1-ffaa-412d-b909-3993be1d78d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.989589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:09.004707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36813c1-ffaa-412d-b909-3993be1d78d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:09.006715 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36813c1-ffaa-412d-b909-3993be1d78d2 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:09.017214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5a9c4-9a14-4dcc-a1c0-1a75496bde9a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:09.089823 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ef3ef-c362-49d5-a3ad-1a2b1176b346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:09.090897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ef3ef-c362-49d5-a3ad-1a2b1176b346 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.091385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ef3ef-c362-49d5-a3ad-1a2b1176b346 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.091877 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ef3ef-c362-49d5-a3ad-1a2b1176b346 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.092901 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ef3ef-c362-49d5-a3ad-1a2b1176b346 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:09.093500 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ef3ef-c362-49d5-a3ad-1a2b1176b346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:09.095150 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ef3ef-c362-49d5-a3ad-1a2b1176b346 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:09.109588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 84/165] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:09.115407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ef3ef-c362-49d5-a3ad-1a2b1176b346 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:09.118409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 82/166] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:21:09.121467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:09.125063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36813c1-ffaa-412d-b909-3993be1d78d2 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:09.239355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36813c1-ffaa-412d-b909-3993be1d78d2 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:09.284183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ef3ef-c362-49d5-a3ad-1a2b1176b346 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:09.287597 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c76db-d0b0-476a-b442-e651c1413ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:09.288681 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c76db-d0b0-476a-b442-e651c1413ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.289131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c76db-d0b0-476a-b442-e651c1413ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.289653 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c76db-d0b0-476a-b442-e651c1413ef4 None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.293165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c76db-d0b0-476a-b442-e651c1413ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:09.294840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c76db-d0b0-476a-b442-e651c1413ef4 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:09.296847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 83/167] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:09 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:09.302813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c76db-d0b0-476a-b442-e651c1413ef4 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:09.346855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fdb0e1-591d-46d0-8830-987fecce9796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:09.348134 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdb0e1-591d-46d0-8830-987fecce9796 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.348633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdb0e1-591d-46d0-8830-987fecce9796 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.349143 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdb0e1-591d-46d0-8830-987fecce9796 None admin] PATH_INFO: `/v3/users/d70de4148c8f4696ac913ae91f5a81cb` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.353861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fdb0e1-591d-46d0-8830-987fecce9796 None admin] RBAC: Authorizing `identity:get_user(user_id=d70de4148c8f4696ac913ae91f5a81cb)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:09.355601 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fdb0e1-591d-46d0-8830-987fecce9796 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:09.357668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 85/168] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:09 2022] GET /identity/v3/users/d70de4148c8f4696ac913ae91f5a81cb => generated 247 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:09.363628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fdb0e1-591d-46d0-8830-987fecce9796 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:09.394401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:09.397633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:09.440362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c76db-d0b0-476a-b442-e651c1413ef4 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:09.482143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839ab00-6463-4f62-9cf7-7818a91fc8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:09.483784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839ab00-6463-4f62-9cf7-7818a91fc8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.484858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839ab00-6463-4f62-9cf7-7818a91fc8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.485493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839ab00-6463-4f62-9cf7-7818a91fc8b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.486697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839ab00-6463-4f62-9cf7-7818a91fc8b5 None admin] RBAC: Adding query filter params (role.id=5addc8ffe20f47749792ac1394f6235c, scope.project.id=d00b803196664866a4fd2eb970b1be0b, user.id=d49ddf0ef68c45e4a61eddb790691a24) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:09.487207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839ab00-6463-4f62-9cf7-7818a91fc8b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:09.488837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839ab00-6463-4f62-9cf7-7818a91fc8b5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:09.496658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 84/169] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:09 2022] GET /identity/v3/role_assignments?user.id=d49ddf0ef68c45e4a61eddb790691a24&scope.project.id=d00b803196664866a4fd2eb970b1be0b&role.id=5addc8ffe20f47749792ac1394f6235c => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:09.498454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b118677-aeb9-4c52-89d3-f525e10b1833 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.499046 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b118677-aeb9-4c52-89d3-f525e10b1833 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.499486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b118677-aeb9-4c52-89d3-f525e10b1833 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.505884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 85/170] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:09.511693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def7aa5-0d90-4ca3-a83a-2264ef23317a None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.512185 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def7aa5-0d90-4ca3-a83a-2264ef23317a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.512603 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def7aa5-0d90-4ca3-a83a-2264ef23317a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.538638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-6def7aa5-0d90-4ca3-a83a-2264ef23317a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:09.540326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6def7aa5-0d90-4ca3-a83a-2264ef23317a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:09.648338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 86/171] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:09.650847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fdb0e1-591d-46d0-8830-987fecce9796 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:09.686133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def7aa5-0d90-4ca3-a83a-2264ef23317a None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:09.758432 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ae442-b6d9-454d-8781-5dab84406f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:09.759162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ae442-b6d9-454d-8781-5dab84406f9f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.759392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ae442-b6d9-454d-8781-5dab84406f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.759673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ae442-b6d9-454d-8781-5dab84406f9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.784662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:09.852865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def7aa5-0d90-4ca3-a83a-2264ef23317a None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:09.881101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ae442-b6d9-454d-8781-5dab84406f9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:09.882600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ae442-b6d9-454d-8781-5dab84406f9f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:09.907886 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:09.934199 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0c6e2-e935-4595-9a4d-41bea53e4281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:09.935455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0c6e2-e935-4595-9a4d-41bea53e4281 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.935972 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0c6e2-e935-4595-9a4d-41bea53e4281 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.936440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0c6e2-e935-4595-9a4d-41bea53e4281 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.956116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0c6e2-e935-4595-9a4d-41bea53e4281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-f9e0c6e2-e935-4595-9a4d-41bea53e4281 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:09.965227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:09.966339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:09.968032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:09.968032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:09.968032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:09.968032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:09.968032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:09.968032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:09.968032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:09.968032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:09.968032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:09.968032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:09.968032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:09.968032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:09.971066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 87/172] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:09.972916 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489728-7ccc-4728-a017-d161a081a0f1 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.973425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489728-7ccc-4728-a017-d161a081a0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.973938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489728-7ccc-4728-a017-d161a081a0f1 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.976011 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 88/173] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:09.977049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca489728-7ccc-4728-a017-d161a081a0f1 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:10.026566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 86/174] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:21:10.028072 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f80c7a-89be-43cd-a2c0-dd1550335e88 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:10.028369 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f80c7a-89be-43cd-a2c0-dd1550335e88 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:10.028654 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f80c7a-89be-43cd-a2c0-dd1550335e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:10.066173 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-63f80c7a-89be-43cd-a2c0-dd1550335e88 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:10.067394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-63f80c7a-89be-43cd-a2c0-dd1550335e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:10.118519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca489728-7ccc-4728-a017-d161a081a0f1 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:10.139391 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:10.161666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571346f9-9769-4283-91eb-579093a7f2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:10.162804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571346f9-9769-4283-91eb-579093a7f2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:10.163289 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571346f9-9769-4283-91eb-579093a7f2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:10.163760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571346f9-9769-4283-91eb-579093a7f2d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:10.164795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571346f9-9769-4283-91eb-579093a7f2d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:10.166072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571346f9-9769-4283-91eb-579093a7f2d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:10.167839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571346f9-9769-4283-91eb-579093a7f2d0 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:10.179995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 89/175] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:10.181378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571346f9-9769-4283-91eb-579093a7f2d0 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:10.191865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 87/176] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:10.192827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f80c7a-89be-43cd-a2c0-dd1550335e88 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:10.269756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:10.302672 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571346f9-9769-4283-91eb-579093a7f2d0 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:10.340045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cc0e2-338a-4c42-9aa6-baa41e6d1e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:10.341331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cc0e2-338a-4c42-9aa6-baa41e6d1e5c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:10.341827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cc0e2-338a-4c42-9aa6-baa41e6d1e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:10.342460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cc0e2-338a-4c42-9aa6-baa41e6d1e5c None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:10.344058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7cc0e2-338a-4c42-9aa6-baa41e6d1e5c None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:10.345838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7cc0e2-338a-4c42-9aa6-baa41e6d1e5c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:10.348004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 90/177] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:10 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:10.349036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cc0e2-338a-4c42-9aa6-baa41e6d1e5c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:10.375329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f80c7a-89be-43cd-a2c0-dd1550335e88 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:10.406333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:10.466402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0621f0d-8669-4898-b4b1-f5d9e10bf1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:10.466810 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0621f0d-8669-4898-b4b1-f5d9e10bf1ea None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:10.467368 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0621f0d-8669-4898-b4b1-f5d9e10bf1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:10.467651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0621f0d-8669-4898-b4b1-f5d9e10bf1ea None admin] PATH_INFO: `/v3/users/d49ddf0ef68c45e4a61eddb790691a24` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:10.469016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0621f0d-8669-4898-b4b1-f5d9e10bf1ea None admin] RBAC: Authorizing `identity:get_user(user_id=d49ddf0ef68c45e4a61eddb790691a24)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:10.470587 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0621f0d-8669-4898-b4b1-f5d9e10bf1ea None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:10.472541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 88/178] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:10 2022] GET /identity/v3/users/d49ddf0ef68c45e4a61eddb790691a24 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:10.477174 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0621f0d-8669-4898-b4b1-f5d9e10bf1ea None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:10.511928 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cc0e2-338a-4c42-9aa6-baa41e6d1e5c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:10.576221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f6d15-c19c-48f9-a1ad-28e96c3b2f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:10.579034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f6d15-c19c-48f9-a1ad-28e96c3b2f47 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:10.579821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f6d15-c19c-48f9-a1ad-28e96c3b2f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:10.580550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f6d15-c19c-48f9-a1ad-28e96c3b2f47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:10.600549 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f6d15-c19c-48f9-a1ad-28e96c3b2f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-cc0f6d15-c19c-48f9-a1ad-28e96c3b2f47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:10.626783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:10.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:10.629558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:10.629558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:10.629558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:10.629558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:10.629558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:10.629558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:10.629558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:10.629558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:10.629558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:10.629558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:10.629558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:10.629558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:10.635910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 91/179] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:10.638284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f6d15-c19c-48f9-a1ad-28e96c3b2f47 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:10.773345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0621f0d-8669-4898-b4b1-f5d9e10bf1ea None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:10.821479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:10.849223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6997cdc-dc8a-4aab-8a11-ef270ae9be50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:10.850048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6997cdc-dc8a-4aab-8a11-ef270ae9be50 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:10.850343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6997cdc-dc8a-4aab-8a11-ef270ae9be50 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:10.850617 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6997cdc-dc8a-4aab-8a11-ef270ae9be50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:10.851780 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6997cdc-dc8a-4aab-8a11-ef270ae9be50 None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=d00b803196664866a4fd2eb970b1be0b, user.id=d70de4148c8f4696ac913ae91f5a81cb) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:10.851898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6997cdc-dc8a-4aab-8a11-ef270ae9be50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:10.853386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6997cdc-dc8a-4aab-8a11-ef270ae9be50 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:10.860871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 89/180] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:10 2022] GET /identity/v3/role_assignments?user.id=d70de4148c8f4696ac913ae91f5a81cb&scope.project.id=d00b803196664866a4fd2eb970b1be0b&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 608 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:10.861783 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6997cdc-dc8a-4aab-8a11-ef270ae9be50 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:10.878401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f6d15-c19c-48f9-a1ad-28e96c3b2f47 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:10.899961 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:10.970937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be220556-f915-4e1a-8864-98f6682e7a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:10.971997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be220556-f915-4e1a-8864-98f6682e7a59 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:10.972714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be220556-f915-4e1a-8864-98f6682e7a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:10.974769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be220556-f915-4e1a-8864-98f6682e7a59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.079723 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6997cdc-dc8a-4aab-8a11-ef270ae9be50 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:11.083507 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be220556-f915-4e1a-8864-98f6682e7a59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:11.085012 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be220556-f915-4e1a-8864-98f6682e7a59 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:11.122301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44670eb9-b79e-4ca5-b195-2e61c252c24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:11.122980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44670eb9-b79e-4ca5-b195-2e61c252c24c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.123318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44670eb9-b79e-4ca5-b195-2e61c252c24c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.124114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44670eb9-b79e-4ca5-b195-2e61c252c24c None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.124861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44670eb9-b79e-4ca5-b195-2e61c252c24c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:11.125193 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44670eb9-b79e-4ca5-b195-2e61c252c24c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:11.126571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44670eb9-b79e-4ca5-b195-2e61c252c24c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:11.142356 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 90/181] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:11.146174 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e41a49-9072-4a7b-a3f4-ca06a7701507 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.146544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e41a49-9072-4a7b-a3f4-ca06a7701507 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.147015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e41a49-9072-4a7b-a3f4-ca06a7701507 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.149782 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 91/182] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:11.152193 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0641e4-abdc-420c-a9e4-a7d07e042c35 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.152726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0641e4-abdc-420c-a9e4-a7d07e042c35 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.154181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0641e4-abdc-420c-a9e4-a7d07e042c35 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.155971 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 92/183] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:11.158763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd2ed8-68cf-4ded-a158-48a56d80f455 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.158763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd2ed8-68cf-4ded-a158-48a56d80f455 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.159160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd2ed8-68cf-4ded-a158-48a56d80f455 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.162711 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 93/184] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:11.163250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd2ed8-68cf-4ded-a158-48a56d80f455 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:11.216869 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 92/185] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:11.218285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb6c2c-a088-4c4f-81e0-4622f3064de6 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.218580 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb6c2c-a088-4c4f-81e0-4622f3064de6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.218861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb6c2c-a088-4c4f-81e0-4622f3064de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.229171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:11.234294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:11.240439 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1acb6c2c-a088-4c4f-81e0-4622f3064de6 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:11.241707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1acb6c2c-a088-4c4f-81e0-4622f3064de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:11.294182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd2ed8-68cf-4ded-a158-48a56d80f455 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:11.320347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 93/186] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:11.321924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8a536-b79f-4a6a-aef0-07b0dac09320 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.322213 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8a536-b79f-4a6a-aef0-07b0dac09320 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.322489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8a536-b79f-4a6a-aef0-07b0dac09320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.343082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1c5a4-7da2-4391-a657-0532ff1646ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:11.343450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1c5a4-7da2-4391-a657-0532ff1646ef None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.343762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1c5a4-7da2-4391-a657-0532ff1646ef None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.344015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1c5a4-7da2-4391-a657-0532ff1646ef None admin] PATH_INFO: `/v3/users/5f71cbdcd01b4e54b50a2286d0e82886` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.351744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-29a8a536-b79f-4a6a-aef0-07b0dac09320 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:11.353545 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-29a8a536-b79f-4a6a-aef0-07b0dac09320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:11.359168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1c5a4-7da2-4391-a657-0532ff1646ef None admin] RBAC: Authorizing `identity:get_user(user_id=5f71cbdcd01b4e54b50a2286d0e82886)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:11.360647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1c5a4-7da2-4391-a657-0532ff1646ef None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:11.362444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 94/187] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:11 2022] GET /identity/v3/users/5f71cbdcd01b4e54b50a2286d0e82886 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:11.363788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616c78f-4c54-40e9-9fb1-00dfce79ed1c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.364182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616c78f-4c54-40e9-9fb1-00dfce79ed1c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.364459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616c78f-4c54-40e9-9fb1-00dfce79ed1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.383756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7616c78f-4c54-40e9-9fb1-00dfce79ed1c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:11.384640 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7616c78f-4c54-40e9-9fb1-00dfce79ed1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:11.425776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 94/188] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:11.426553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8a536-b79f-4a6a-aef0-07b0dac09320 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:11.476716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 95/189] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:11.478171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616c78f-4c54-40e9-9fb1-00dfce79ed1c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:11.514606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:11.523282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:11.563607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8a536-b79f-4a6a-aef0-07b0dac09320 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:11.597402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616c78f-4c54-40e9-9fb1-00dfce79ed1c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:11.613636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f012d62-0ccb-4c7f-9c60-36cea9f37ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:11.614996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f012d62-0ccb-4c7f-9c60-36cea9f37ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.615402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f012d62-0ccb-4c7f-9c60-36cea9f37ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.615796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f012d62-0ccb-4c7f-9c60-36cea9f37ed0 None admin] PATH_INFO: `/v3/projects/7b2c588cb1484b8caf109e0c625cc831` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.617245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f012d62-0ccb-4c7f-9c60-36cea9f37ed0 None admin] RBAC: Authorizing `identity:get_project(project_id=7b2c588cb1484b8caf109e0c625cc831)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:11.619168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f012d62-0ccb-4c7f-9c60-36cea9f37ed0 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:11.621036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 95/190] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:11 2022] GET /identity/v3/projects/7b2c588cb1484b8caf109e0c625cc831 => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:11.626884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f012d62-0ccb-4c7f-9c60-36cea9f37ed0 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:11.645829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c81e4-8cf9-49c1-be53-66f73fa98d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:11.646629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c81e4-8cf9-49c1-be53-66f73fa98d9b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.646912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c81e4-8cf9-49c1-be53-66f73fa98d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.647189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c81e4-8cf9-49c1-be53-66f73fa98d9b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.658381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c81e4-8cf9-49c1-be53-66f73fa98d9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-302c81e4-8cf9-49c1-be53-66f73fa98d9b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:11.666148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:11.667208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 96/191] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:11.668201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c81e4-8cf9-49c1-be53-66f73fa98d9b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:11.735845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f012d62-0ccb-4c7f-9c60-36cea9f37ed0 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:11.756098 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:11.770428 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5b50f-f6b9-4427-b579-04c7db412f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:11.771078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5b50f-f6b9-4427-b579-04c7db412f21 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.771364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5b50f-f6b9-4427-b579-04c7db412f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.771564 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c81e4-8cf9-49c1-be53-66f73fa98d9b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:11.771658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5b50f-f6b9-4427-b579-04c7db412f21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.774321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:11.806937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3498e-b9b7-46eb-8e46-f43569366aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:11.807760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3498e-b9b7-46eb-8e46-f43569366aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.808039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3498e-b9b7-46eb-8e46-f43569366aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.808311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3498e-b9b7-46eb-8e46-f43569366aa7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.819482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3498e-b9b7-46eb-8e46-f43569366aa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-1db3498e-b9b7-46eb-8e46-f43569366aa7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:11.827636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:11.828787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:11.829607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:11.829607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:11.829607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:11.829607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:11.829607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:11.829607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:11.829607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:11.829607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:11.829607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:11.829607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:11.829607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:11.829607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:11.829607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 97/192] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:21:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:11.830270 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3498e-b9b7-46eb-8e46-f43569366aa7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:11.862334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5b50f-f6b9-4427-b579-04c7db412f21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:11.863855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5b50f-f6b9-4427-b579-04c7db412f21 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:11.933789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3498e-b9b7-46eb-8e46-f43569366aa7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:11.970844 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341759f3-1c2c-4dbe-ad7a-5a114008cb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:11.971640 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341759f3-1c2c-4dbe-ad7a-5a114008cb3b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.971928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341759f3-1c2c-4dbe-ad7a-5a114008cb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.972246 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341759f3-1c2c-4dbe-ad7a-5a114008cb3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.973414 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341759f3-1c2c-4dbe-ad7a-5a114008cb3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:11.974922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341759f3-1c2c-4dbe-ad7a-5a114008cb3b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:11.975684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 96/193] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:11.977063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5b50f-f6b9-4427-b579-04c7db412f21 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:11.980044 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 98/194] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:11.981993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341759f3-1c2c-4dbe-ad7a-5a114008cb3b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:11.998313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.013290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.094406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5b50f-f6b9-4427-b579-04c7db412f21 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.097890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341759f3-1c2c-4dbe-ad7a-5a114008cb3b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.128992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ae614-3b2b-4699-ab7c-78853208c270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.129675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ae614-3b2b-4699-ab7c-78853208c270 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.129956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ae614-3b2b-4699-ab7c-78853208c270 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.130229 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ae614-3b2b-4699-ab7c-78853208c270 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.131056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ae614-3b2b-4699-ab7c-78853208c270 None admin] RBAC: Adding query filter params (role.id=2bdabfafa05e443e842fe34d5f03e2c9, scope.project.id=7b2c588cb1484b8caf109e0c625cc831, user.id=d49ddf0ef68c45e4a61eddb790691a24) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:12.131366 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ae614-3b2b-4699-ab7c-78853208c270 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.132811 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ae614-3b2b-4699-ab7c-78853208c270 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.138669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 97/195] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:11 2022] GET /identity/v3/role_assignments?user.id=d49ddf0ef68c45e4a61eddb790691a24&scope.project.id=7b2c588cb1484b8caf109e0c625cc831&role.id=2bdabfafa05e443e842fe34d5f03e2c9 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:12.142194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ae614-3b2b-4699-ab7c-78853208c270 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.142853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ebcc2d-13f8-4597-bbd6-04579f023ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.143548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebcc2d-13f8-4597-bbd6-04579f023ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.143837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebcc2d-13f8-4597-bbd6-04579f023ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.144120 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebcc2d-13f8-4597-bbd6-04579f023ae9 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.144887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ebcc2d-13f8-4597-bbd6-04579f023ae9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:12.145230 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ebcc2d-13f8-4597-bbd6-04579f023ae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.146641 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ebcc2d-13f8-4597-bbd6-04579f023ae9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.156638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 99/196] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:12.157641 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ebcc2d-13f8-4597-bbd6-04579f023ae9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.256573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.285056 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ebcc2d-13f8-4597-bbd6-04579f023ae9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.290569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.295389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ae614-3b2b-4699-ab7c-78853208c270 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.325051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebc276-89be-4c0c-8682-14eddd488bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.325973 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebc276-89be-4c0c-8682-14eddd488bcf None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.326258 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebc276-89be-4c0c-8682-14eddd488bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.327094 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebc276-89be-4c0c-8682-14eddd488bcf None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.328730 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebc276-89be-4c0c-8682-14eddd488bcf None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.330228 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebc276-89be-4c0c-8682-14eddd488bcf None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.332045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 100/197] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:12.332785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebc276-89be-4c0c-8682-14eddd488bcf None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.337031 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c96a5-23f0-4ab0-acda-cc6c54276243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.337742 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c96a5-23f0-4ab0-acda-cc6c54276243 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.338031 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c96a5-23f0-4ab0-acda-cc6c54276243 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.338310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c96a5-23f0-4ab0-acda-cc6c54276243 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.339077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c96a5-23f0-4ab0-acda-cc6c54276243 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:12.339391 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c96a5-23f0-4ab0-acda-cc6c54276243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.340788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c96a5-23f0-4ab0-acda-cc6c54276243 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.350226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 98/198] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:12.351077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c96a5-23f0-4ab0-acda-cc6c54276243 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.485039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c96a5-23f0-4ab0-acda-cc6c54276243 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.487119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebc276-89be-4c0c-8682-14eddd488bcf None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.523102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111c353-e9a7-4c78-a29c-1540fc649927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.523326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8befc84-0ede-485a-b881-cb64f3ac2a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.523769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111c353-e9a7-4c78-a29c-1540fc649927 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.523965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111c353-e9a7-4c78-a29c-1540fc649927 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.524056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8befc84-0ede-485a-b881-cb64f3ac2a0a None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.524237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111c353-e9a7-4c78-a29c-1540fc649927 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.524343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8befc84-0ede-485a-b881-cb64f3ac2a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.524513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8befc84-0ede-485a-b881-cb64f3ac2a0a None admin] PATH_INFO: `/v3/users/9aa008bceb70450a92b10af150f8e8a8` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.524945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111c353-e9a7-4c78-a29c-1540fc649927 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.525787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8befc84-0ede-485a-b881-cb64f3ac2a0a None admin] RBAC: Authorizing `identity:get_user(user_id=9aa008bceb70450a92b10af150f8e8a8)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.526414 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111c353-e9a7-4c78-a29c-1540fc649927 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.527232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8befc84-0ede-485a-b881-cb64f3ac2a0a None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.528917 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 99/199] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/users/9aa008bceb70450a92b10af150f8e8a8 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:12.529802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8befc84-0ede-485a-b881-cb64f3ac2a0a None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.544720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.550160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 101/200] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:21:12 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:12.563075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111c353-e9a7-4c78-a29c-1540fc649927 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.570852 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.663879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8befc84-0ede-485a-b881-cb64f3ac2a0a None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.699716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111c353-e9a7-4c78-a29c-1540fc649927 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.705076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cff19c-cce3-41d7-bdb1-2c4af0f0ca31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.705076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cff19c-cce3-41d7-bdb1-2c4af0f0ca31 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.705554 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cff19c-cce3-41d7-bdb1-2c4af0f0ca31 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.705767 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cff19c-cce3-41d7-bdb1-2c4af0f0ca31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.706592 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cff19c-cce3-41d7-bdb1-2c4af0f0ca31 None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b, user.id=5f71cbdcd01b4e54b50a2286d0e82886) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:12.706908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cff19c-cce3-41d7-bdb1-2c4af0f0ca31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.708416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cff19c-cce3-41d7-bdb1-2c4af0f0ca31 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.725423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 100/201] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/role_assignments?user.id=5f71cbdcd01b4e54b50a2286d0e82886&scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:12.725423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cff19c-cce3-41d7-bdb1-2c4af0f0ca31 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.778068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2ffa9-eb38-4f1b-8c37-accd9e85581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.778890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2ffa9-eb38-4f1b-8c37-accd9e85581d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.779164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2ffa9-eb38-4f1b-8c37-accd9e85581d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.779596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2ffa9-eb38-4f1b-8c37-accd9e85581d None admin] PATH_INFO: `/v3/users/5f71cbdcd01b4e54b50a2286d0e82886` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.781990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2ffa9-eb38-4f1b-8c37-accd9e85581d None admin] RBAC: Authorizing `identity:get_user(user_id=5f71cbdcd01b4e54b50a2286d0e82886)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.784212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2ffa9-eb38-4f1b-8c37-accd9e85581d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.784978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 102/202] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/users/5f71cbdcd01b4e54b50a2286d0e82886 => generated 283 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:12.791343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2ffa9-eb38-4f1b-8c37-accd9e85581d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.857972 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.868666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cff19c-cce3-41d7-bdb1-2c4af0f0ca31 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.872929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.925511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2ffa9-eb38-4f1b-8c37-accd9e85581d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.930848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a1eb6-43ba-4fbc-b32f-e0c68b747811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.931719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a1eb6-43ba-4fbc-b32f-e0c68b747811 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.932078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a1eb6-43ba-4fbc-b32f-e0c68b747811 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.932432 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a1eb6-43ba-4fbc-b32f-e0c68b747811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.975506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06fe44-e92d-4626-a5ba-2856a25f3880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.976294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06fe44-e92d-4626-a5ba-2856a25f3880 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.976650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06fe44-e92d-4626-a5ba-2856a25f3880 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.977001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06fe44-e92d-4626-a5ba-2856a25f3880 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:13.039245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a1eb6-43ba-4fbc-b32f-e0c68b747811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:13.041292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a1eb6-43ba-4fbc-b32f-e0c68b747811 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:13.074902 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06fe44-e92d-4626-a5ba-2856a25f3880 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:13.076746 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06fe44-e92d-4626-a5ba-2856a25f3880 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:13.154269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:13.173151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:13.185037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 101/203] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:13.191730 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a1eb6-43ba-4fbc-b32f-e0c68b747811 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:13.244231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 103/204] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:13.250013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06fe44-e92d-4626-a5ba-2856a25f3880 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:13.313353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a1eb6-43ba-4fbc-b32f-e0c68b747811 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:13.380588 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993167b1-80bd-4748-a6f0-e6ae5e77d885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:13.381512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993167b1-80bd-4748-a6f0-e6ae5e77d885 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:13.381798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993167b1-80bd-4748-a6f0-e6ae5e77d885 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:13.382075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993167b1-80bd-4748-a6f0-e6ae5e77d885 None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:13.383660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993167b1-80bd-4748-a6f0-e6ae5e77d885 None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:13.385423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993167b1-80bd-4748-a6f0-e6ae5e77d885 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:13.389300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 102/205] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:13 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:13.400990 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993167b1-80bd-4748-a6f0-e6ae5e77d885 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:13.463627 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06fe44-e92d-4626-a5ba-2856a25f3880 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:13.521473 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:13.535602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bd3ae-c17a-434e-874b-3e65dd73e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:13.536478 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bd3ae-c17a-434e-874b-3e65dd73e39b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:13.536762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bd3ae-c17a-434e-874b-3e65dd73e39b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:13.537139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bd3ae-c17a-434e-874b-3e65dd73e39b None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:13.542893 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6bd3ae-c17a-434e-874b-3e65dd73e39b None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:13.544378 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6bd3ae-c17a-434e-874b-3e65dd73e39b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:13.547064 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 104/206] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:13 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:13.552433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bd3ae-c17a-434e-874b-3e65dd73e39b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:13.601054 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993167b1-80bd-4748-a6f0-e6ae5e77d885 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:13.622959 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:13.679307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63933dfc-fa2a-4a40-8cbb-154c79cea19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:13.681475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933dfc-fa2a-4a40-8cbb-154c79cea19b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:13.682755 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933dfc-fa2a-4a40-8cbb-154c79cea19b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:13.683481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933dfc-fa2a-4a40-8cbb-154c79cea19b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:13.686328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63933dfc-fa2a-4a40-8cbb-154c79cea19b None admin] RBAC: Adding query filter params (role.id=6a4f5f6d39ed47f4a5036b99c7386949, scope.project.id=d00b803196664866a4fd2eb970b1be0b, user.id=9aa008bceb70450a92b10af150f8e8a8) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:13.688295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63933dfc-fa2a-4a40-8cbb-154c79cea19b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:13.692184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63933dfc-fa2a-4a40-8cbb-154c79cea19b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:13.712408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 103/207] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:13 2022] GET /identity/v3/role_assignments?user.id=9aa008bceb70450a92b10af150f8e8a8&scope.project.id=d00b803196664866a4fd2eb970b1be0b&role.id=6a4f5f6d39ed47f4a5036b99c7386949 => generated 608 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:13.746065 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bd3ae-c17a-434e-874b-3e65dd73e39b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:13.802855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001bcf3-d7a3-49f5-92a1-af94f7410407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:13.803912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001bcf3-d7a3-49f5-92a1-af94f7410407 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:13.804418 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001bcf3-d7a3-49f5-92a1-af94f7410407 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:13.804913 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001bcf3-d7a3-49f5-92a1-af94f7410407 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:13.806068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001bcf3-d7a3-49f5-92a1-af94f7410407 None admin] RBAC: Adding query filter params (role.id=5addc8ffe20f47749792ac1394f6235c, scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b, user.id=5f71cbdcd01b4e54b50a2286d0e82886) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:13.806655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001bcf3-d7a3-49f5-92a1-af94f7410407 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:13.808394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001bcf3-d7a3-49f5-92a1-af94f7410407 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:13.816554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 105/208] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:13 2022] GET /identity/v3/role_assignments?user.id=5f71cbdcd01b4e54b50a2286d0e82886&scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b&role.id=5addc8ffe20f47749792ac1394f6235c => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:15.278739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c76d5-93e4-4066-a135-107c49cffc31 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.279222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c76d5-93e4-4066-a135-107c49cffc31 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.279222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c76d5-93e4-4066-a135-107c49cffc31 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.281291 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 104/209] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:15.282925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25038ec-7052-4abd-b62b-aab4e4224fae None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.283348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25038ec-7052-4abd-b62b-aab4e4224fae None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.283555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25038ec-7052-4abd-b62b-aab4e4224fae None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.285190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 106/210] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:15.289920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edeb921-5590-4f7b-b0d6-b8b8041e96f9 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.290275 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50277a32-71e6-4d64-8b37-b5ae2ea1f44e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.290565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50277a32-71e6-4d64-8b37-b5ae2ea1f44e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.290845 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50277a32-71e6-4d64-8b37-b5ae2ea1f44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.291111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edeb921-5590-4f7b-b0d6-b8b8041e96f9 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.291601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edeb921-5590-4f7b-b0d6-b8b8041e96f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.308447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-50277a32-71e6-4d64-8b37-b5ae2ea1f44e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:15.309674 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-50277a32-71e6-4d64-8b37-b5ae2ea1f44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:15.310225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-3edeb921-5590-4f7b-b0d6-b8b8041e96f9 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:15.312368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-3edeb921-5590-4f7b-b0d6-b8b8041e96f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:15.393144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 107/211] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:15.402048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 105/212] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:15.416896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50277a32-71e6-4d64-8b37-b5ae2ea1f44e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:15.430431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edeb921-5590-4f7b-b0d6-b8b8041e96f9 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:15.457108 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:15.501584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:15.540986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50277a32-71e6-4d64-8b37-b5ae2ea1f44e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:15.569963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edeb921-5590-4f7b-b0d6-b8b8041e96f9 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:15.581736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8481d40-e37f-48f2-a2fd-55d5ad4efd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:15.582544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8481d40-e37f-48f2-a2fd-55d5ad4efd32 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.582823 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8481d40-e37f-48f2-a2fd-55d5ad4efd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.583099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8481d40-e37f-48f2-a2fd-55d5ad4efd32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.594481 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8481d40-e37f-48f2-a2fd-55d5ad4efd32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f8481d40-e37f-48f2-a2fd-55d5ad4efd32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:15.602206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:15.603739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:15.604570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:15.604570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:15.604570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:15.604570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:15.604570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:15.604570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:15.604570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:15.604570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:15.604570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:15.604570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:15.604570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:15.604570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:15.604570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 108/213] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:15.605801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af71b77-eb5c-4ed7-9cf2-0b22ab444467 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.605801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af71b77-eb5c-4ed7-9cf2-0b22ab444467 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.605801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af71b77-eb5c-4ed7-9cf2-0b22ab444467 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.606705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 109/214] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:15.611859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af71b77-eb5c-4ed7-9cf2-0b22ab444467 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:15.619610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0134c2c-e72f-420d-9211-be79025511e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:15.620422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0134c2c-e72f-420d-9211-be79025511e8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.620706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0134c2c-e72f-420d-9211-be79025511e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.621501 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0134c2c-e72f-420d-9211-be79025511e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.622720 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0134c2c-e72f-420d-9211-be79025511e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:15.625811 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0134c2c-e72f-420d-9211-be79025511e8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:15.631602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 106/215] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:15.631602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb7561-261a-41aa-a63f-f49986ab15c9 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.631602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb7561-261a-41aa-a63f-f49986ab15c9 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.631602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb7561-261a-41aa-a63f-f49986ab15c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.651369 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-a9eb7561-261a-41aa-a63f-f49986ab15c9 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:15.652549 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a9eb7561-261a-41aa-a63f-f49986ab15c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:15.726188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:15.741140 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 107/216] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:15.741140 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb7561-261a-41aa-a63f-f49986ab15c9 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:15.748261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af71b77-eb5c-4ed7-9cf2-0b22ab444467 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:15.790426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43f88c-c45b-4af2-acb1-e62b7cc6c92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:15.791154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43f88c-c45b-4af2-acb1-e62b7cc6c92f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.791375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43f88c-c45b-4af2-acb1-e62b7cc6c92f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.791649 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43f88c-c45b-4af2-acb1-e62b7cc6c92f None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.792391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43f88c-c45b-4af2-acb1-e62b7cc6c92f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:15.792696 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43f88c-c45b-4af2-acb1-e62b7cc6c92f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:15.794101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43f88c-c45b-4af2-acb1-e62b7cc6c92f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:15.803680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:15.808824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 110/217] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:15.808824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43f88c-c45b-4af2-acb1-e62b7cc6c92f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:15.879285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb7561-261a-41aa-a63f-f49986ab15c9 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:15.932407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba9600-b6c9-415c-8ccb-c168930b261d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:15.933523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba9600-b6c9-415c-8ccb-c168930b261d None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.933859 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba9600-b6c9-415c-8ccb-c168930b261d None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.934173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba9600-b6c9-415c-8ccb-c168930b261d None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.934998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba9600-b6c9-415c-8ccb-c168930b261d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:15.936970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba9600-b6c9-415c-8ccb-c168930b261d None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:15.939299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43f88c-c45b-4af2-acb1-e62b7cc6c92f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:15.962718 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 108/218] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:21:15 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:15.963568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba9600-b6c9-415c-8ccb-c168930b261d None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:15.999962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f8a67-d6d4-4061-99c0-b12c4447a265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.001290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f8a67-d6d4-4061-99c0-b12c4447a265 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.001861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f8a67-d6d4-4061-99c0-b12c4447a265 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.002384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f8a67-d6d4-4061-99c0-b12c4447a265 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.016902 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f8a67-d6d4-4061-99c0-b12c4447a265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-af2f8a67-d6d4-4061-99c0-b12c4447a265 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:16.026330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:16.027344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:16.028166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:16.028166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:16.028166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:16.028166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:16.028166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:16.028166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:16.028166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:16.028166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:16.028166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:16.028166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:16.028166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:16.028166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:16.036772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 111/219] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:16.038631 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f8a67-d6d4-4061-99c0-b12c4447a265 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.058204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:16.125396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba9600-b6c9-415c-8ccb-c168930b261d None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.135807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:16.175717 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744ca2b-b82b-4ab9-b602-02e5ad912ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.176627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744ca2b-b82b-4ab9-b602-02e5ad912ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.176912 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744ca2b-b82b-4ab9-b602-02e5ad912ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.177226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744ca2b-b82b-4ab9-b602-02e5ad912ec1 None admin] PATH_INFO: `/v3/users/5f71cbdcd01b4e54b50a2286d0e82886` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.178789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744ca2b-b82b-4ab9-b602-02e5ad912ec1 None admin] RBAC: Authorizing `identity:get_user(user_id=5f71cbdcd01b4e54b50a2286d0e82886)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:16.180313 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744ca2b-b82b-4ab9-b602-02e5ad912ec1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:16.187024 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 109/220] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:15 2022] GET /identity/v3/users/5f71cbdcd01b4e54b50a2286d0e82886 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:16.188085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744ca2b-b82b-4ab9-b602-02e5ad912ec1 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.191584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f8a67-d6d4-4061-99c0-b12c4447a265 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.243209 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e078d-9eaf-4b64-8c03-ecbd73e45795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.244756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e078d-9eaf-4b64-8c03-ecbd73e45795 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.245373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e078d-9eaf-4b64-8c03-ecbd73e45795 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.245857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e078d-9eaf-4b64-8c03-ecbd73e45795 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.247239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e078d-9eaf-4b64-8c03-ecbd73e45795 None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:16.247779 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e078d-9eaf-4b64-8c03-ecbd73e45795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:16.250205 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e078d-9eaf-4b64-8c03-ecbd73e45795 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:16.263110 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 112/221] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:16.270022 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e078d-9eaf-4b64-8c03-ecbd73e45795 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.330926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744ca2b-b82b-4ab9-b602-02e5ad912ec1 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.372312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f324e-1de7-4ccb-86bc-a91dba080d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.373306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f324e-1de7-4ccb-86bc-a91dba080d85 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.373767 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f324e-1de7-4ccb-86bc-a91dba080d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.374219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f324e-1de7-4ccb-86bc-a91dba080d85 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.375719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:16.376222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f324e-1de7-4ccb-86bc-a91dba080d85 None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:16.377487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f324e-1de7-4ccb-86bc-a91dba080d85 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:16.380170 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 110/222] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:16 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:16.386096 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f324e-1de7-4ccb-86bc-a91dba080d85 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.408985 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e078d-9eaf-4b64-8c03-ecbd73e45795 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.446925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:16.457727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def76d55-7bba-4972-862c-5f314a3422aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.458630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def76d55-7bba-4972-862c-5f314a3422aa None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.458912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def76d55-7bba-4972-862c-5f314a3422aa None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.459185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def76d55-7bba-4972-862c-5f314a3422aa None admin] PATH_INFO: `/v3/users/6fb522b33b6a46319815df813dd33fb9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.478504 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def76d55-7bba-4972-862c-5f314a3422aa None admin] RBAC: Authorizing `identity:get_user(user_id=6fb522b33b6a46319815df813dd33fb9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:16.480285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def76d55-7bba-4972-862c-5f314a3422aa None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:16.486486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 113/223] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:16 2022] GET /identity/v3/users/6fb522b33b6a46319815df813dd33fb9 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:16.486486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd5dc3-978a-4acb-97b3-8fe4c6aedf99 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.486486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd5dc3-978a-4acb-97b3-8fe4c6aedf99 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.486486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd5dc3-978a-4acb-97b3-8fe4c6aedf99 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.487021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 114/224] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:16.492190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a5129-5c12-4f8d-8ca8-99e9cecffa41 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.492586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a5129-5c12-4f8d-8ca8-99e9cecffa41 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.492963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a5129-5c12-4f8d-8ca8-99e9cecffa41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.511581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b73a5129-5c12-4f8d-8ca8-99e9cecffa41 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:16.512960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b73a5129-5c12-4f8d-8ca8-99e9cecffa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:16.528003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f324e-1de7-4ccb-86bc-a91dba080d85 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.580436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af977de6-4b60-4d65-b331-46a60c31f77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.581918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977de6-4b60-4d65-b331-46a60c31f77f None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.582774 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977de6-4b60-4d65-b331-46a60c31f77f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.583540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977de6-4b60-4d65-b331-46a60c31f77f None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b/users/5f71cbdcd01b4e54b50a2286d0e82886/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.586654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af977de6-4b60-4d65-b331-46a60c31f77f None admin] RBAC: Authorizing `identity:create_grant(project_id=a49ddfabaeee47e0ad65b7bf6470e03b, user_id=5f71cbdcd01b4e54b50a2286d0e82886, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:16.588496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af977de6-4b60-4d65-b331-46a60c31f77f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:16.590585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 115/225] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:16.591587 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a5129-5c12-4f8d-8ca8-99e9cecffa41 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.619943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 111/226] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:21:16 2022] PUT /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b/users/5f71cbdcd01b4e54b50a2286d0e82886/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:16.621652 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af977de6-4b60-4d65-b331-46a60c31f77f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.727754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:16.800811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a5129-5c12-4f8d-8ca8-99e9cecffa41 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.830645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af977de6-4b60-4d65-b331-46a60c31f77f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.833855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:16.893525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9efb22-d6d8-4649-b5be-48882913b5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.898934 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9efb22-d6d8-4649-b5be-48882913b5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.899569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9efb22-d6d8-4649-b5be-48882913b5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.902948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9efb22-d6d8-4649-b5be-48882913b5c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.912271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a054a-6e7e-4775-a746-fe7b05ed7e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.912947 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a054a-6e7e-4775-a746-fe7b05ed7e43 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.913311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a054a-6e7e-4775-a746-fe7b05ed7e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.913609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a054a-6e7e-4775-a746-fe7b05ed7e43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.920987 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9efb22-d6d8-4649-b5be-48882913b5c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-bb9efb22-d6d8-4649-b5be-48882913b5c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:16.933585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:16.934530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:16.935384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:16.935384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:16.935384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:16.935384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:16.935384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:16.935384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:16.935384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:16.935384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:16.935384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:16.935384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:16.935384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:16.935384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:16.938735 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 112/227] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:21:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:16.946011 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9efb22-d6d8-4649-b5be-48882913b5c7 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:17.005877 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a054a-6e7e-4775-a746-fe7b05ed7e43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:17.007935 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a054a-6e7e-4775-a746-fe7b05ed7e43 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:17.121967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9efb22-d6d8-4649-b5be-48882913b5c7 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:17.139129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:17.181116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e491d-5564-4842-af8f-c323741e0603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:17.181116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e491d-5564-4842-af8f-c323741e0603 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:17.181116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e491d-5564-4842-af8f-c323741e0603 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:17.181116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e491d-5564-4842-af8f-c323741e0603 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:17.181520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e491d-5564-4842-af8f-c323741e0603 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:17.181520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e491d-5564-4842-af8f-c323741e0603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:17.181520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e491d-5564-4842-af8f-c323741e0603 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:17.192094 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 113/228] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:17.198029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e491d-5564-4842-af8f-c323741e0603 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:17.232991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:17.254765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 116/229] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:17.266213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a054a-6e7e-4775-a746-fe7b05ed7e43 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:17.321645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e491d-5564-4842-af8f-c323741e0603 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:17.360938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4408a-7266-48e3-9b9b-c2cbdd9bae2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:17.361859 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4408a-7266-48e3-9b9b-c2cbdd9bae2c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:17.362311 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4408a-7266-48e3-9b9b-c2cbdd9bae2c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:17.362594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4408a-7266-48e3-9b9b-c2cbdd9bae2c None admin] PATH_INFO: `/v3/users/5f71cbdcd01b4e54b50a2286d0e82886` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:17.363852 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4408a-7266-48e3-9b9b-c2cbdd9bae2c None admin] RBAC: Authorizing `identity:get_user(user_id=5f71cbdcd01b4e54b50a2286d0e82886)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:17.365658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4408a-7266-48e3-9b9b-c2cbdd9bae2c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:17.367344 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 114/230] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:17 2022] GET /identity/v3/users/5f71cbdcd01b4e54b50a2286d0e82886 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:17.373897 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4408a-7266-48e3-9b9b-c2cbdd9bae2c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:17.498406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4408a-7266-48e3-9b9b-c2cbdd9bae2c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:17.518037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:17.539566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02616875-26f3-4a8a-939c-dd23a0f3c117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:17.540438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616875-26f3-4a8a-939c-dd23a0f3c117 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:17.540728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616875-26f3-4a8a-939c-dd23a0f3c117 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:17.541010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616875-26f3-4a8a-939c-dd23a0f3c117 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:17.542421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02616875-26f3-4a8a-939c-dd23a0f3c117 None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:17.544278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02616875-26f3-4a8a-939c-dd23a0f3c117 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:17.546916 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 115/231] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:17 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:17.552583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02616875-26f3-4a8a-939c-dd23a0f3c117 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:17.623739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a054a-6e7e-4775-a746-fe7b05ed7e43 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:17.668224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02616875-26f3-4a8a-939c-dd23a0f3c117 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:17.708123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e27b6-a6f8-4398-9262-e6ff6646931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:17.709038 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e27b6-a6f8-4398-9262-e6ff6646931a None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:17.709486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e27b6-a6f8-4398-9262-e6ff6646931a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:17.709778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e27b6-a6f8-4398-9262-e6ff6646931a None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b/users/5f71cbdcd01b4e54b50a2286d0e82886/roles/5addc8ffe20f47749792ac1394f6235c` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:17.712103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e27b6-a6f8-4398-9262-e6ff6646931a None admin] RBAC: Authorizing `identity:create_grant(project_id=a49ddfabaeee47e0ad65b7bf6470e03b, user_id=5f71cbdcd01b4e54b50a2286d0e82886, role_id=5addc8ffe20f47749792ac1394f6235c)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:17.713622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e27b6-a6f8-4398-9262-e6ff6646931a None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:17.734341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 116/232] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:21:17 2022] PUT /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b/users/5f71cbdcd01b4e54b50a2286d0e82886/roles/5addc8ffe20f47749792ac1394f6235c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:17.740714 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51246d0-56d9-496f-8f7e-23c127343433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:17.741678 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51246d0-56d9-496f-8f7e-23c127343433 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:17.741973 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51246d0-56d9-496f-8f7e-23c127343433 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:17.742232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51246d0-56d9-496f-8f7e-23c127343433 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:17.743706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51246d0-56d9-496f-8f7e-23c127343433 None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:17.745317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51246d0-56d9-496f-8f7e-23c127343433 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:17.747276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 117/233] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:17 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 482 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:17.754591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e27b6-a6f8-4398-9262-e6ff6646931a None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:17.786642 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:17.901835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e27b6-a6f8-4398-9262-e6ff6646931a None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:17.959648 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41610b98-c671-495a-a7f6-a3d437a222a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:17.960726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610b98-c671-495a-a7f6-a3d437a222a5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:17.961285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610b98-c671-495a-a7f6-a3d437a222a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:17.961775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610b98-c671-495a-a7f6-a3d437a222a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:17.962828 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41610b98-c671-495a-a7f6-a3d437a222a5 None admin] RBAC: Adding query filter params (role.id=2bdabfafa05e443e842fe34d5f03e2c9, scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b, user.id=6fb522b33b6a46319815df813dd33fb9) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:17.963353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41610b98-c671-495a-a7f6-a3d437a222a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:17.965077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41610b98-c671-495a-a7f6-a3d437a222a5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:17.977949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 117/234] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:17 2022] GET /identity/v3/role_assignments?user.id=6fb522b33b6a46319815df813dd33fb9&scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b&role.id=2bdabfafa05e443e842fe34d5f03e2c9 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:18.930478 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1582c99-64c7-4fd8-bfb8-bb6879145340 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:18.931264 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1582c99-64c7-4fd8-bfb8-bb6879145340 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:18.931264 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1582c99-64c7-4fd8-bfb8-bb6879145340 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:18.932420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 118/235] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:18.939710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae91a7-bbc9-421a-91d9-b2f4d71ca766 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:18.941132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae91a7-bbc9-421a-91d9-b2f4d71ca766 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:18.941889 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae91a7-bbc9-421a-91d9-b2f4d71ca766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:18.976594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-fdae91a7-bbc9-421a-91d9-b2f4d71ca766 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:18.979092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-fdae91a7-bbc9-421a-91d9-b2f4d71ca766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:19.101476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 118/236] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:19.136748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1582c99-64c7-4fd8-bfb8-bb6879145340 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:19.186317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:19.272602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80f4c1-ce6c-40b2-8854-70c606245a8a None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:19.273601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80f4c1-ce6c-40b2-8854-70c606245a8a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:19.274315 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80f4c1-ce6c-40b2-8854-70c606245a8a None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:19.284370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 119/237] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:19.286215 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1582c99-64c7-4fd8-bfb8-bb6879145340 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:19.288069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a5323-4865-4c33-979a-cb8b4d6a5897 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:19.288875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a5323-4865-4c33-979a-cb8b4d6a5897 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:19.289676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a5323-4865-4c33-979a-cb8b4d6a5897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:19.338091 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-4e0a5323-4865-4c33-979a-cb8b4d6a5897 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:19.339315 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4e0a5323-4865-4c33-979a-cb8b4d6a5897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:19.348037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ae601-e26e-4258-bc51-9d5c6d5d1418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:19.348037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ae601-e26e-4258-bc51-9d5c6d5d1418 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:19.348037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ae601-e26e-4258-bc51-9d5c6d5d1418 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:19.348037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ae601-e26e-4258-bc51-9d5c6d5d1418 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:19.370807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:19.372542 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ae601-e26e-4258-bc51-9d5c6d5d1418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-8c3ae601-e26e-4258-bc51-9d5c6d5d1418 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:19.385862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:19.387663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:19.390756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:19.390756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:19.390756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:19.390756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:19.390756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:19.390756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:19.390756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:19.390756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:19.390756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:19.390756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:19.390756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:19.390756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:19.391863 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 119/238] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:19.399316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ae601-e26e-4258-bc51-9d5c6d5d1418 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:19.500454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 120/239] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:19.550116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a5323-4865-4c33-979a-cb8b4d6a5897 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:19.612879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:19.627056 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ae601-e26e-4258-bc51-9d5c6d5d1418 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:19.667870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca43edb-8d0c-49a2-a248-ec71ce037789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:19.669505 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca43edb-8d0c-49a2-a248-ec71ce037789 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:19.670141 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca43edb-8d0c-49a2-a248-ec71ce037789 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:19.670687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca43edb-8d0c-49a2-a248-ec71ce037789 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:19.672181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca43edb-8d0c-49a2-a248-ec71ce037789 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:19.672960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca43edb-8d0c-49a2-a248-ec71ce037789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:19.687383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca43edb-8d0c-49a2-a248-ec71ce037789 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:19.711484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 120/240] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:19.719906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffdfc3-5cf5-4969-909c-f3d79949a8e1 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:19.720981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffdfc3-5cf5-4969-909c-f3d79949a8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:19.721811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffdfc3-5cf5-4969-909c-f3d79949a8e1 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:19.724765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 121/241] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:19.727282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffdfc3-5cf5-4969-909c-f3d79949a8e1 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:19.773228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a5323-4865-4c33-979a-cb8b4d6a5897 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:19.827016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55269d7a-c2fd-4b93-b7dd-3b77eabe9fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:19.829156 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55269d7a-c2fd-4b93-b7dd-3b77eabe9fad None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:19.829780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55269d7a-c2fd-4b93-b7dd-3b77eabe9fad None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:19.830557 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55269d7a-c2fd-4b93-b7dd-3b77eabe9fad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:19.832914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55269d7a-c2fd-4b93-b7dd-3b77eabe9fad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:19.836133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55269d7a-c2fd-4b93-b7dd-3b77eabe9fad None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:19.839937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 121/242] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:19.848048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904e3f9-bdb1-4291-808b-16d9b6349924 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:19.849116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904e3f9-bdb1-4291-808b-16d9b6349924 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:19.849534 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904e3f9-bdb1-4291-808b-16d9b6349924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:19.910288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffdfc3-5cf5-4969-909c-f3d79949a8e1 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:19.915863 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e904e3f9-bdb1-4291-808b-16d9b6349924 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:19.918456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e904e3f9-bdb1-4291-808b-16d9b6349924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:19.939684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:19.952262 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cccd68-598d-4529-a4e1-ea4c5e719c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:19.953575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cccd68-598d-4529-a4e1-ea4c5e719c58 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:19.954047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cccd68-598d-4529-a4e1-ea4c5e719c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:19.954626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cccd68-598d-4529-a4e1-ea4c5e719c58 None admin] PATH_INFO: `/v3/users/7c77983a5fc443ed916b8af81f638df6` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:19.971934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cccd68-598d-4529-a4e1-ea4c5e719c58 None admin] RBAC: Authorizing `identity:get_user(user_id=7c77983a5fc443ed916b8af81f638df6)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:19.973811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cccd68-598d-4529-a4e1-ea4c5e719c58 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:19.977054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 122/243] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:19 2022] GET /identity/v3/users/7c77983a5fc443ed916b8af81f638df6 => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:19.978051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cccd68-598d-4529-a4e1-ea4c5e719c58 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:19.994503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:20.072912 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 122/244] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:20.075853 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904e3f9-bdb1-4291-808b-16d9b6349924 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:20.151071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cccd68-598d-4529-a4e1-ea4c5e719c58 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:20.230160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904e3f9-bdb1-4291-808b-16d9b6349924 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:20.238347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187df3e-fa16-42c9-9b41-f30d14e6f3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:20.239398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187df3e-fa16-42c9-9b41-f30d14e6f3e7 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:20.239905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187df3e-fa16-42c9-9b41-f30d14e6f3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:20.240376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187df3e-fa16-42c9-9b41-f30d14e6f3e7 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:20.241698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187df3e-fa16-42c9-9b41-f30d14e6f3e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:20.243431 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187df3e-fa16-42c9-9b41-f30d14e6f3e7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:20.281304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 123/245] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:21:19 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:20.281304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187df3e-fa16-42c9-9b41-f30d14e6f3e7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:20.342113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6b136-25c1-48ae-bddd-78da9fbb880c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:20.342707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6b136-25c1-48ae-bddd-78da9fbb880c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:20.343006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6b136-25c1-48ae-bddd-78da9fbb880c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:20.343286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6b136-25c1-48ae-bddd-78da9fbb880c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:20.386192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:20.455655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:20.524139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6b136-25c1-48ae-bddd-78da9fbb880c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:20.525975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6b136-25c1-48ae-bddd-78da9fbb880c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:20.548602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187df3e-fa16-42c9-9b41-f30d14e6f3e7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:20.653463 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5328498-c7e0-48a2-a47c-88c5c9b323a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:20.654834 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5328498-c7e0-48a2-a47c-88c5c9b323a7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:20.655336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5328498-c7e0-48a2-a47c-88c5c9b323a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:20.655716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5328498-c7e0-48a2-a47c-88c5c9b323a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:20.661753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 123/246] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:20.667203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6b136-25c1-48ae-bddd-78da9fbb880c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:20.686934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5328498-c7e0-48a2-a47c-88c5c9b323a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-e5328498-c7e0-48a2-a47c-88c5c9b323a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:20.694879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:20.695788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:20.697026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:20.697026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:20.697026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:20.697026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:20.697026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:20.697026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:20.697026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:20.697026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:20.697026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:20.697026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:20.697026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:20.697026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:20.697026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 124/247] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:20.718153 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5328498-c7e0-48a2-a47c-88c5c9b323a7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:20.772583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:20.910621 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6b136-25c1-48ae-bddd-78da9fbb880c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:20.939270 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5328498-c7e0-48a2-a47c-88c5c9b323a7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:20.989565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:20.999263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824341ca-dc81-43f2-965f-66b9748a296b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:21.000804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824341ca-dc81-43f2-965f-66b9748a296b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:21.001419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824341ca-dc81-43f2-965f-66b9748a296b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:21.001906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824341ca-dc81-43f2-965f-66b9748a296b None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:21.003849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824341ca-dc81-43f2-965f-66b9748a296b None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:21.005757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824341ca-dc81-43f2-965f-66b9748a296b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:21.007623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b272aeb-56ee-44d3-bc71-09dee065f70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:21.008773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 124/248] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:20 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:21.008891 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b272aeb-56ee-44d3-bc71-09dee065f70b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:21.010620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a6f96-353b-4e38-8e79-8e980f916cd5 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:21.011189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a6f96-353b-4e38-8e79-8e980f916cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:21.011683 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a6f96-353b-4e38-8e79-8e980f916cd5 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:21.011795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b272aeb-56ee-44d3-bc71-09dee065f70b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:21.012336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b272aeb-56ee-44d3-bc71-09dee065f70b None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:21.013520 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b272aeb-56ee-44d3-bc71-09dee065f70b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:21.013851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b272aeb-56ee-44d3-bc71-09dee065f70b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:21.014715 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 125/249] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:21.015691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b272aeb-56ee-44d3-bc71-09dee065f70b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:21.021059 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a6f96-353b-4e38-8e79-8e980f916cd5 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:21.027076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 125/250] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:21.031722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e6d04-abcb-4597-9a6f-dd0a816a0998 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:21.032139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e6d04-abcb-4597-9a6f-dd0a816a0998 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:21.032323 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e6d04-abcb-4597-9a6f-dd0a816a0998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:21.052223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-fd9e6d04-abcb-4597-9a6f-dd0a816a0998 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:21.053431 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fd9e6d04-abcb-4597-9a6f-dd0a816a0998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:21.161484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 126/251] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:21.162312 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e6d04-abcb-4597-9a6f-dd0a816a0998 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:21.202884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a6f96-353b-4e38-8e79-8e980f916cd5 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:21.219065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:21.244944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501680b-9b23-47d6-a051-6585b8e53dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:21.246847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501680b-9b23-47d6-a051-6585b8e53dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:21.248478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501680b-9b23-47d6-a051-6585b8e53dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:21.249500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501680b-9b23-47d6-a051-6585b8e53dd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:21.250719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501680b-9b23-47d6-a051-6585b8e53dd7 None admin] RBAC: Adding query filter params (role.id=6a4f5f6d39ed47f4a5036b99c7386949, scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b, user.id=7c77983a5fc443ed916b8af81f638df6) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:21.251576 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501680b-9b23-47d6-a051-6585b8e53dd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:21.253497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501680b-9b23-47d6-a051-6585b8e53dd7 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:21.261575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 126/252] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:21 2022] GET /identity/v3/role_assignments?user.id=7c77983a5fc443ed916b8af81f638df6&scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b&role.id=6a4f5f6d39ed47f4a5036b99c7386949 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:21.263248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8786b-1caf-4b34-9318-8d35892a4b90 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:21.263828 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8786b-1caf-4b34-9318-8d35892a4b90 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:21.264200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8786b-1caf-4b34-9318-8d35892a4b90 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:21.266220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 127/253] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:21.271298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8786b-1caf-4b34-9318-8d35892a4b90 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:21.318461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e6d04-abcb-4597-9a6f-dd0a816a0998 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:21.344342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:21.367620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e04ac-7921-487d-89ad-779bca207e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:21.368876 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e04ac-7921-487d-89ad-779bca207e4a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:21.369380 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e04ac-7921-487d-89ad-779bca207e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:21.369890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e04ac-7921-487d-89ad-779bca207e4a None admin] PATH_INFO: `/v3/users/5f71cbdcd01b4e54b50a2286d0e82886` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:21.371669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e04ac-7921-487d-89ad-779bca207e4a None admin] RBAC: Authorizing `identity:get_user(user_id=5f71cbdcd01b4e54b50a2286d0e82886)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:21.373661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e04ac-7921-487d-89ad-779bca207e4a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:21.375799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 127/254] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:21 2022] GET /identity/v3/users/5f71cbdcd01b4e54b50a2286d0e82886 => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:21.377658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1260b-f47a-4010-9d83-abd309c998ff None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:21.378131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1260b-f47a-4010-9d83-abd309c998ff None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:21.378585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1260b-f47a-4010-9d83-abd309c998ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:21.398342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2db1260b-f47a-4010-9d83-abd309c998ff None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:21.399869 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2db1260b-f47a-4010-9d83-abd309c998ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:21.488608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 128/255] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:21.489760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1260b-f47a-4010-9d83-abd309c998ff None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:21.541855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8786b-1caf-4b34-9318-8d35892a4b90 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:21.626332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98a0be-a054-4162-8038-338ce323180a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:21.627414 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98a0be-a054-4162-8038-338ce323180a None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:21.627845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98a0be-a054-4162-8038-338ce323180a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:21.628295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98a0be-a054-4162-8038-338ce323180a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:21.646720 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98a0be-a054-4162-8038-338ce323180a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-5c98a0be-a054-4162-8038-338ce323180a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:21.655951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:21.657150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:21.658123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:21.658123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:21.658123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:21.658123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:21.658123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:21.658123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:21.658123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:21.658123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:21.658123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:21.658123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:21.658123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:21.658123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:21.665303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 128/256] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:21:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:21.666676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98a0be-a054-4162-8038-338ce323180a None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:21.709825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:21.731278 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1260b-f47a-4010-9d83-abd309c998ff None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:21.773790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:21.813389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e597a-0531-4373-b21c-914fef7a63c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:21.814322 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e597a-0531-4373-b21c-914fef7a63c9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:21.814783 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e597a-0531-4373-b21c-914fef7a63c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:21.815211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e597a-0531-4373-b21c-914fef7a63c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:21.852772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98a0be-a054-4162-8038-338ce323180a None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:21.962347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e597a-0531-4373-b21c-914fef7a63c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:21.964923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e597a-0531-4373-b21c-914fef7a63c9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:21.981357 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de6756-d605-46aa-9730-160249851e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:21.983057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de6756-d605-46aa-9730-160249851e28 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:21.983462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de6756-d605-46aa-9730-160249851e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:21.983833 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de6756-d605-46aa-9730-160249851e28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:22.030597 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de6756-d605-46aa-9730-160249851e28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-44de6756-d605-46aa-9730-160249851e28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:22.038596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:22.040620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 129/257] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:22.042884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de6756-d605-46aa-9730-160249851e28 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:22.204107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 129/258] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:22.213733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e597a-0531-4373-b21c-914fef7a63c9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:22.269964 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:22.323143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:22.330640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de6756-d605-46aa-9730-160249851e28 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:22.372334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859b20e-2659-43cf-9f79-98b9816ec286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:22.373389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859b20e-2659-43cf-9f79-98b9816ec286 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:22.373781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859b20e-2659-43cf-9f79-98b9816ec286 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:22.374070 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859b20e-2659-43cf-9f79-98b9816ec286 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:22.374872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859b20e-2659-43cf-9f79-98b9816ec286 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:22.375188 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859b20e-2659-43cf-9f79-98b9816ec286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:22.376678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859b20e-2659-43cf-9f79-98b9816ec286 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:22.387293 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 130/259] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:22.388368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859b20e-2659-43cf-9f79-98b9816ec286 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:22.522763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859b20e-2659-43cf-9f79-98b9816ec286 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:22.531481 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e597a-0531-4373-b21c-914fef7a63c9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:22.564808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b298f01-dd1d-439a-bdfc-a0d335a91bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:22.565879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298f01-dd1d-439a-bdfc-a0d335a91bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:22.566171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298f01-dd1d-439a-bdfc-a0d335a91bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:22.566455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298f01-dd1d-439a-bdfc-a0d335a91bd5 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:22.567847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b298f01-dd1d-439a-bdfc-a0d335a91bd5 None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:22.569422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b298f01-dd1d-439a-bdfc-a0d335a91bd5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:22.571580 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 131/260] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:22 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:22.572574 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b298f01-dd1d-439a-bdfc-a0d335a91bd5 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:22.621676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:22.669129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ea716-588a-4bb4-89b2-bd4443c2f100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:22.670131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ea716-588a-4bb4-89b2-bd4443c2f100 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:22.670294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ea716-588a-4bb4-89b2-bd4443c2f100 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:22.670460 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ea716-588a-4bb4-89b2-bd4443c2f100 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:22.678152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ea716-588a-4bb4-89b2-bd4443c2f100 None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:22.678468 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ea716-588a-4bb4-89b2-bd4443c2f100 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:22.679895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ea716-588a-4bb4-89b2-bd4443c2f100 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:22.696838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b298f01-dd1d-439a-bdfc-a0d335a91bd5 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:22.707563 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 130/261] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 495 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:22.708458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ea716-588a-4bb4-89b2-bd4443c2f100 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:22.778335 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222cb31-046d-4c5c-aec5-84b6872270c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:22.781376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222cb31-046d-4c5c-aec5-84b6872270c1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:22.782434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222cb31-046d-4c5c-aec5-84b6872270c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:22.784413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222cb31-046d-4c5c-aec5-84b6872270c1 None admin] PATH_INFO: `/v3/users/5f71cbdcd01b4e54b50a2286d0e82886` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:22.788745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222cb31-046d-4c5c-aec5-84b6872270c1 None admin] RBAC: Authorizing `identity:get_user(user_id=5f71cbdcd01b4e54b50a2286d0e82886)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:22.793236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222cb31-046d-4c5c-aec5-84b6872270c1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:22.800075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 132/262] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:22 2022] GET /identity/v3/users/5f71cbdcd01b4e54b50a2286d0e82886 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:22.803487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222cb31-046d-4c5c-aec5-84b6872270c1 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:22.965935 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:23.006948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ea716-588a-4bb4-89b2-bd4443c2f100 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:23.066707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825843d-f6cb-4838-9cfc-f026b158cb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:23.067735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825843d-f6cb-4838-9cfc-f026b158cb13 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:23.068213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825843d-f6cb-4838-9cfc-f026b158cb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:23.068850 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825843d-f6cb-4838-9cfc-f026b158cb13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:23.070514 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825843d-f6cb-4838-9cfc-f026b158cb13 None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b, user.id=5f71cbdcd01b4e54b50a2286d0e82886) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:23.071047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825843d-f6cb-4838-9cfc-f026b158cb13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:23.074181 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:23.075196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825843d-f6cb-4838-9cfc-f026b158cb13 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:23.089321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 131/263] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:22 2022] GET /identity/v3/role_assignments?user.id=5f71cbdcd01b4e54b50a2286d0e82886&scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 608 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:23.090707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825843d-f6cb-4838-9cfc-f026b158cb13 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:23.092710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222cb31-046d-4c5c-aec5-84b6872270c1 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:23.219538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbabb6d-411b-43e9-a6a9-acdb6a21c8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:23.220996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbabb6d-411b-43e9-a6a9-acdb6a21c8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:23.221506 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbabb6d-411b-43e9-a6a9-acdb6a21c8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:23.222013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbabb6d-411b-43e9-a6a9-acdb6a21c8a7 None admin] PATH_INFO: `/v3/users/6fb522b33b6a46319815df813dd33fb9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:23.230559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbabb6d-411b-43e9-a6a9-acdb6a21c8a7 None admin] RBAC: Authorizing `identity:get_user(user_id=6fb522b33b6a46319815df813dd33fb9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:23.233835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbabb6d-411b-43e9-a6a9-acdb6a21c8a7 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:23.237036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 133/264] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:22 2022] GET /identity/v3/users/6fb522b33b6a46319815df813dd33fb9 => generated 297 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:23.258458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbabb6d-411b-43e9-a6a9-acdb6a21c8a7 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:23.274344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825843d-f6cb-4838-9cfc-f026b158cb13 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:23.319967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39da918-9051-4dd1-a2d9-f9fcf6b0d451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:23.320656 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39da918-9051-4dd1-a2d9-f9fcf6b0d451 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:23.320942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39da918-9051-4dd1-a2d9-f9fcf6b0d451 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:23.321265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39da918-9051-4dd1-a2d9-f9fcf6b0d451 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:23.336902 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:23.408429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39da918-9051-4dd1-a2d9-f9fcf6b0d451 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:23.410254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39da918-9051-4dd1-a2d9-f9fcf6b0d451 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:23.550616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 132/265] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:23.561556 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39da918-9051-4dd1-a2d9-f9fcf6b0d451 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:23.581853 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbabb6d-411b-43e9-a6a9-acdb6a21c8a7 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:23.652899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c283df-641f-42f0-94c0-3e93076fc942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:23.653944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c283df-641f-42f0-94c0-3e93076fc942 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:23.654771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c283df-641f-42f0-94c0-3e93076fc942 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:23.655051 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c283df-641f-42f0-94c0-3e93076fc942 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:23.658211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c283df-641f-42f0-94c0-3e93076fc942 None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:23.660178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c283df-641f-42f0-94c0-3e93076fc942 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:23.663087 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 134/266] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:23 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:23.667554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:23.670299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c283df-641f-42f0-94c0-3e93076fc942 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:23.737395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:23.752847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39da918-9051-4dd1-a2d9-f9fcf6b0d451 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:23.793225 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2075d-5fb2-4b4d-87ce-88b2d463883b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:23.794039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2075d-5fb2-4b4d-87ce-88b2d463883b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:23.794324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2075d-5fb2-4b4d-87ce-88b2d463883b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:23.794590 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2075d-5fb2-4b4d-87ce-88b2d463883b None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:23.796406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2075d-5fb2-4b4d-87ce-88b2d463883b None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:23.797906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2075d-5fb2-4b4d-87ce-88b2d463883b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:23.800161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 133/267] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:23 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:23.808456 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2075d-5fb2-4b4d-87ce-88b2d463883b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:23.886505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c283df-641f-42f0-94c0-3e93076fc942 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:23.965741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf4827-cf6f-46e7-9b49-93e0241ad634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:23.967440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf4827-cf6f-46e7-9b49-93e0241ad634 None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:23.967760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf4827-cf6f-46e7-9b49-93e0241ad634 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:23.968082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf4827-cf6f-46e7-9b49-93e0241ad634 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b/users/6fb522b33b6a46319815df813dd33fb9/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:23.970409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf4827-cf6f-46e7-9b49-93e0241ad634 None admin] RBAC: Authorizing `identity:create_grant(project_id=a49ddfabaeee47e0ad65b7bf6470e03b, user_id=6fb522b33b6a46319815df813dd33fb9, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:23.971999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf4827-cf6f-46e7-9b49-93e0241ad634 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:23.986764 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2075d-5fb2-4b4d-87ce-88b2d463883b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:23.997135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 135/268] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:21:23 2022] PUT /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b/users/6fb522b33b6a46319815df813dd33fb9/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:24.010575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:24.051796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaad5be-5969-4440-900c-668a96f0cd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:24.052594 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaad5be-5969-4440-900c-668a96f0cd00 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:24.052879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaad5be-5969-4440-900c-668a96f0cd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:24.053218 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaad5be-5969-4440-900c-668a96f0cd00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:24.054167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaad5be-5969-4440-900c-668a96f0cd00 None admin] RBAC: Adding query filter params (role.id=5addc8ffe20f47749792ac1394f6235c, scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b, user.id=5f71cbdcd01b4e54b50a2286d0e82886) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:24.054485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaad5be-5969-4440-900c-668a96f0cd00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:24.056038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaad5be-5969-4440-900c-668a96f0cd00 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:24.068902 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 134/269] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:23 2022] GET /identity/v3/role_assignments?user.id=5f71cbdcd01b4e54b50a2286d0e82886&scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b&role.id=5addc8ffe20f47749792ac1394f6235c => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:24.309030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9cf5c-6bd7-4104-8101-144295fae4b0 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:24.309804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9cf5c-6bd7-4104-8101-144295fae4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:24.310917 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9cf5c-6bd7-4104-8101-144295fae4b0 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:24.313060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 136/270] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:24.320164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77956606-1b3a-4600-8e26-6d86c7577db5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:24.320648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77956606-1b3a-4600-8e26-6d86c7577db5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:24.320648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77956606-1b3a-4600-8e26-6d86c7577db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:24.341363 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-77956606-1b3a-4600-8e26-6d86c7577db5 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:24.343224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-77956606-1b3a-4600-8e26-6d86c7577db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:24.433173 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 135/271] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:24.461119 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9cf5c-6bd7-4104-8101-144295fae4b0 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:24.594111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:24.609263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9cf5c-6bd7-4104-8101-144295fae4b0 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:24.651650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2fd0a-edc2-4747-ac81-02b6fa3bf610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:24.652883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2fd0a-edc2-4747-ac81-02b6fa3bf610 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:24.653440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2fd0a-edc2-4747-ac81-02b6fa3bf610 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:24.654098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2fd0a-edc2-4747-ac81-02b6fa3bf610 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:24.656243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2fd0a-edc2-4747-ac81-02b6fa3bf610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:24.658039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2fd0a-edc2-4747-ac81-02b6fa3bf610 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:24.660242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 137/272] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:24.666285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77956606-1b3a-4600-8e26-6d86c7577db5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:24.687929 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38283d04-4a6f-4743-887e-8c6d7016cdb2 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:24.688403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38283d04-4a6f-4743-887e-8c6d7016cdb2 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:24.688882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38283d04-4a6f-4743-887e-8c6d7016cdb2 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:24.693184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 138/273] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:24.700625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1e157-ca70-4317-8266-ee59ccd9a5bf None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:24.701792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1e157-ca70-4317-8266-ee59ccd9a5bf None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:24.702250 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1e157-ca70-4317-8266-ee59ccd9a5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:24.731508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-5ea1e157-ca70-4317-8266-ee59ccd9a5bf None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:24.734339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5ea1e157-ca70-4317-8266-ee59ccd9a5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:24.823570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77956606-1b3a-4600-8e26-6d86c7577db5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:24.828296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:24.876973 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 139/274] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:24.903004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553200e-ef3c-4805-93b1-80e01030b7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:24.904626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553200e-ef3c-4805-93b1-80e01030b7e9 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:24.905367 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553200e-ef3c-4805-93b1-80e01030b7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:24.905871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553200e-ef3c-4805-93b1-80e01030b7e9 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:24.906809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553200e-ef3c-4805-93b1-80e01030b7e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:24.908505 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553200e-ef3c-4805-93b1-80e01030b7e9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:24.922820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1e157-ca70-4317-8266-ee59ccd9a5bf None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:24.936855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 136/275] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:21:24 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:25.106915 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:25.132770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1e157-ca70-4317-8266-ee59ccd9a5bf None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:25.182992 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b339d25-8350-4128-85b3-5dd0a9e48423 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:25.183421 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b339d25-8350-4128-85b3-5dd0a9e48423 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:25.183785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b339d25-8350-4128-85b3-5dd0a9e48423 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:25.185611 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 137/276] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:21:25.195375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cddd6-3478-48b5-95e0-179709f043bb None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:25.196410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cddd6-3478-48b5-95e0-179709f043bb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:25.196884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cddd6-3478-48b5-95e0-179709f043bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:25.206019 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f407901-d015-46f2-ba42-e22c8aa80c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:25.207621 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f407901-d015-46f2-ba42-e22c8aa80c14 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:25.208126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f407901-d015-46f2-ba42-e22c8aa80c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:25.208640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f407901-d015-46f2-ba42-e22c8aa80c14 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:25.231619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-176cddd6-3478-48b5-95e0-179709f043bb None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:25.231884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f407901-d015-46f2-ba42-e22c8aa80c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:25.232011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-176cddd6-3478-48b5-95e0-179709f043bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-0f407901-d015-46f2-ba42-e22c8aa80c14 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:25.244572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:25.246461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:25.248010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:25.248010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:25.248010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:25.248010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:25.248010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:25.248010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:25.248010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:25.248010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:25.248010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:25.248010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:25.248010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:25.248010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:25.248010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 140/277] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:25.255219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f407901-d015-46f2-ba42-e22c8aa80c14 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:25.397402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 138/278] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:25.435852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cddd6-3478-48b5-95e0-179709f043bb None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:25.451825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f407901-d015-46f2-ba42-e22c8aa80c14 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:25.519753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de158464-10fa-45c0-a091-2e1c352f64b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:25.520445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158464-10fa-45c0-a091-2e1c352f64b1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:25.520731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158464-10fa-45c0-a091-2e1c352f64b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:25.521009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158464-10fa-45c0-a091-2e1c352f64b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:25.521837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de158464-10fa-45c0-a091-2e1c352f64b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:25.522154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de158464-10fa-45c0-a091-2e1c352f64b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:25.523612 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de158464-10fa-45c0-a091-2e1c352f64b1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:25.534541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 141/279] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:25.541518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de158464-10fa-45c0-a091-2e1c352f64b1 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:25.557878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:25.625153 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:25.667872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cddd6-3478-48b5-95e0-179709f043bb None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:25.706736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de158464-10fa-45c0-a091-2e1c352f64b1 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:25.755194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef5832-ab2f-4af6-b16f-c349e4f81eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:25.756659 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef5832-ab2f-4af6-b16f-c349e4f81eae None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:25.757177 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef5832-ab2f-4af6-b16f-c349e4f81eae None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:25.757674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef5832-ab2f-4af6-b16f-c349e4f81eae None admin] PATH_INFO: `/v3/users/7c77983a5fc443ed916b8af81f638df6` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:25.759216 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef5832-ab2f-4af6-b16f-c349e4f81eae None admin] RBAC: Authorizing `identity:get_user(user_id=7c77983a5fc443ed916b8af81f638df6)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:25.761513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef5832-ab2f-4af6-b16f-c349e4f81eae None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:25.763683 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 142/280] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:25 2022] GET /identity/v3/users/7c77983a5fc443ed916b8af81f638df6 => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:25.769894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef5832-ab2f-4af6-b16f-c349e4f81eae None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:25.790573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffd07c-6948-438c-979d-f150ed044ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:25.791891 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffd07c-6948-438c-979d-f150ed044ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:25.792600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffd07c-6948-438c-979d-f150ed044ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:25.793132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffd07c-6948-438c-979d-f150ed044ae4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:25.818907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffd07c-6948-438c-979d-f150ed044ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-0bffd07c-6948-438c-979d-f150ed044ae4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:25.828436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:25.829461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:25.830249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:25.830249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:25.830249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:25.830249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:25.830249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:25.830249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:25.830249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:25.830249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:25.830249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:25.830249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:25.830249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:25.830249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:25.843521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 139/281] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:25.849621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffd07c-6948-438c-979d-f150ed044ae4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:25.969523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:25.987313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffd07c-6948-438c-979d-f150ed044ae4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:26.029831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9c6c7-9e52-4ceb-ab71-dea41558fcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:26.030916 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9c6c7-9e52-4ceb-ab71-dea41558fcba None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:26.031410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9c6c7-9e52-4ceb-ab71-dea41558fcba None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:26.031914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9c6c7-9e52-4ceb-ab71-dea41558fcba None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:26.032897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9c6c7-9e52-4ceb-ab71-dea41558fcba None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:26.033526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9c6c7-9e52-4ceb-ab71-dea41558fcba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:26.035252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9c6c7-9e52-4ceb-ab71-dea41558fcba None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:26.045753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 140/282] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:26.049128 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef5832-ab2f-4af6-b16f-c349e4f81eae None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:26.051967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9c6c7-9e52-4ceb-ab71-dea41558fcba None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:26.068627 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:26.138015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ee75a-1f30-467e-b659-b689c06396d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:26.140753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ee75a-1f30-467e-b659-b689c06396d3 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:26.141292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ee75a-1f30-467e-b659-b689c06396d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:26.142035 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ee75a-1f30-467e-b659-b689c06396d3 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:26.143965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ee75a-1f30-467e-b659-b689c06396d3 None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:26.146641 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ee75a-1f30-467e-b659-b689c06396d3 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:26.149800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 143/283] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:25 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:26.162802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ee75a-1f30-467e-b659-b689c06396d3 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:26.177033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9c6c7-9e52-4ceb-ab71-dea41558fcba None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:26.217274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8c3d3-20ca-49ee-98d6-fc7d6aff2106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:26.218577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8c3d3-20ca-49ee-98d6-fc7d6aff2106 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:26.219072 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8c3d3-20ca-49ee-98d6-fc7d6aff2106 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:26.219611 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8c3d3-20ca-49ee-98d6-fc7d6aff2106 None admin] PATH_INFO: `/v3/users/55a15f45d2534b4e80c552fe0b8133cd` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:26.235930 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8c3d3-20ca-49ee-98d6-fc7d6aff2106 None admin] RBAC: Authorizing `identity:get_user(user_id=55a15f45d2534b4e80c552fe0b8133cd)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:26.237908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8c3d3-20ca-49ee-98d6-fc7d6aff2106 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:26.240395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 141/284] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:26 2022] GET /identity/v3/users/55a15f45d2534b4e80c552fe0b8133cd => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:26.246453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8c3d3-20ca-49ee-98d6-fc7d6aff2106 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:26.347586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:26.407598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8c3d3-20ca-49ee-98d6-fc7d6aff2106 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:26.418922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ee75a-1f30-467e-b659-b689c06396d3 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:26.447158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154ef4a-23ef-48d5-abfb-ea9521c666a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:26.448351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154ef4a-23ef-48d5-abfb-ea9521c666a9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:26.448899 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154ef4a-23ef-48d5-abfb-ea9521c666a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:26.449411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154ef4a-23ef-48d5-abfb-ea9521c666a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:26.450481 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154ef4a-23ef-48d5-abfb-ea9521c666a9 None admin] RBAC: Adding query filter params (role.id=2bdabfafa05e443e842fe34d5f03e2c9, user.id=55a15f45d2534b4e80c552fe0b8133cd, scope.system=all) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:26.451099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154ef4a-23ef-48d5-abfb-ea9521c666a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:26.453247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154ef4a-23ef-48d5-abfb-ea9521c666a9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:26.459934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 142/285] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Jun 7 08:21:26 2022] GET /identity/v3/role_assignments?user.id=55a15f45d2534b4e80c552fe0b8133cd&scope.system=all&role.id=2bdabfafa05e443e842fe34d5f03e2c9 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:26.494154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4977db1-0dcf-4d2b-9f66-7c817be9af8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:26.495016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4977db1-0dcf-4d2b-9f66-7c817be9af8f None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:26.495300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4977db1-0dcf-4d2b-9f66-7c817be9af8f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:26.495573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4977db1-0dcf-4d2b-9f66-7c817be9af8f None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b/users/7c77983a5fc443ed916b8af81f638df6/roles/6a4f5f6d39ed47f4a5036b99c7386949` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:26.498591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4977db1-0dcf-4d2b-9f66-7c817be9af8f None admin] RBAC: Authorizing `identity:create_grant(project_id=a49ddfabaeee47e0ad65b7bf6470e03b, user_id=7c77983a5fc443ed916b8af81f638df6, role_id=6a4f5f6d39ed47f4a5036b99c7386949)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:26.500274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4977db1-0dcf-4d2b-9f66-7c817be9af8f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:26.517719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 144/286] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:21:26 2022] PUT /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b/users/7c77983a5fc443ed916b8af81f638df6/roles/6a4f5f6d39ed47f4a5036b99c7386949 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:26.743813 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc043d-db5c-4ae3-b981-7ae87ce4c403 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:26.744281 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc043d-db5c-4ae3-b981-7ae87ce4c403 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:26.744383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc043d-db5c-4ae3-b981-7ae87ce4c403 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:26.746274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 143/287] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:26.753832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc00656-2e6e-4dec-b5ba-16b2345b3996 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:26.754347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc00656-2e6e-4dec-b5ba-16b2345b3996 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:26.754810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc00656-2e6e-4dec-b5ba-16b2345b3996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:26.783122 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:26.792421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-6fc00656-2e6e-4dec-b5ba-16b2345b3996 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:26.796463 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6fc00656-2e6e-4dec-b5ba-16b2345b3996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:26.931065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 145/288] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:26.958179 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc043d-db5c-4ae3-b981-7ae87ce4c403 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:27.117428 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc043d-db5c-4ae3-b981-7ae87ce4c403 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:27.202762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83e65a-062a-48eb-b282-fb931d63bfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:27.204404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83e65a-062a-48eb-b282-fb931d63bfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:27.204902 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83e65a-062a-48eb-b282-fb931d63bfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:27.205405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83e65a-062a-48eb-b282-fb931d63bfe0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:27.212185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:27.238312 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83e65a-062a-48eb-b282-fb931d63bfe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-5c83e65a-062a-48eb-b282-fb931d63bfe0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:27.258738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:27.260690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:27.262754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:27.262754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:27.262754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:27.262754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:27.262754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:27.262754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:27.262754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:27.262754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:27.262754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:27.262754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:27.262754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:27.262754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:27.262754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 144/289] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:27.269326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc00656-2e6e-4dec-b5ba-16b2345b3996 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:27.427651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc00656-2e6e-4dec-b5ba-16b2345b3996 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:27.487082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3357c-9154-4d62-9ca4-dabcb38140f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:27.487780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3357c-9154-4d62-9ca4-dabcb38140f3 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:27.488065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3357c-9154-4d62-9ca4-dabcb38140f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:27.488342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3357c-9154-4d62-9ca4-dabcb38140f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:27.489127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3357c-9154-4d62-9ca4-dabcb38140f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:27.489442 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3357c-9154-4d62-9ca4-dabcb38140f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:27.490849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3357c-9154-4d62-9ca4-dabcb38140f3 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:27.494469 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:27.504732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 146/290] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:27.508098 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83e65a-062a-48eb-b282-fb931d63bfe0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:27.636847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83e65a-062a-48eb-b282-fb931d63bfe0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:27.678405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b337b83-a62c-42dc-b1fb-68323cc2e74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:27.679219 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b337b83-a62c-42dc-b1fb-68323cc2e74a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:27.679501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b337b83-a62c-42dc-b1fb-68323cc2e74a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:27.679781 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b337b83-a62c-42dc-b1fb-68323cc2e74a None admin] PATH_INFO: `/v3/users/6fb522b33b6a46319815df813dd33fb9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:27.681073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b337b83-a62c-42dc-b1fb-68323cc2e74a None admin] RBAC: Authorizing `identity:get_user(user_id=6fb522b33b6a46319815df813dd33fb9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:27.682575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b337b83-a62c-42dc-b1fb-68323cc2e74a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:27.684615 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 145/291] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:27 2022] GET /identity/v3/users/6fb522b33b6a46319815df813dd33fb9 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:27.690392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3357c-9154-4d62-9ca4-dabcb38140f3 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:27.904777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3357c-9154-4d62-9ca4-dabcb38140f3 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:27.969054 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da51d6-6912-4b4f-8b0e-e1a9e436f6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:27.970079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da51d6-6912-4b4f-8b0e-e1a9e436f6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:27.970583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da51d6-6912-4b4f-8b0e-e1a9e436f6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:27.971069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da51d6-6912-4b4f-8b0e-e1a9e436f6a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:28.107228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:28.112239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da51d6-6912-4b4f-8b0e-e1a9e436f6a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:28.115244 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da51d6-6912-4b4f-8b0e-e1a9e436f6a5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:28.314848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 147/292] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:28.324091 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b337b83-a62c-42dc-b1fb-68323cc2e74a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:28.459078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:28.537546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b337b83-a62c-42dc-b1fb-68323cc2e74a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:28.596085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe940b-c370-47d1-b205-b541ea4480a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:28.597359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe940b-c370-47d1-b205-b541ea4480a6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:28.597851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe940b-c370-47d1-b205-b541ea4480a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:28.598335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe940b-c370-47d1-b205-b541ea4480a6 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:28.605349 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe940b-c370-47d1-b205-b541ea4480a6 None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:28.607488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe940b-c370-47d1-b205-b541ea4480a6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:28.610033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 146/293] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:28 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:28.619277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da51d6-6912-4b4f-8b0e-e1a9e436f6a5 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:28.811548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4437ff9-9377-4783-b8bd-767700c46292 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:28.812353 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4437ff9-9377-4783-b8bd-767700c46292 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:28.812874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4437ff9-9377-4783-b8bd-767700c46292 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:28.814545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 147/294] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:28.820423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36859796-2df0-4790-94c5-079b269b4f60 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:28.821065 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36859796-2df0-4790-94c5-079b269b4f60 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:28.821600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36859796-2df0-4790-94c5-079b269b4f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:28.833507 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:28.840074 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da51d6-6912-4b4f-8b0e-e1a9e436f6a5 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:28.841029 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-36859796-2df0-4790-94c5-079b269b4f60 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:28.843686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-36859796-2df0-4790-94c5-079b269b4f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:28.906406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a2769-9fb6-4a37-ac84-5e0dd301487c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:28.907700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a2769-9fb6-4a37-ac84-5e0dd301487c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:28.908175 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a2769-9fb6-4a37-ac84-5e0dd301487c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:28.908645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a2769-9fb6-4a37-ac84-5e0dd301487c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:28.910271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a2769-9fb6-4a37-ac84-5e0dd301487c None admin] RBAC: Adding query filter params (role.id=2bdabfafa05e443e842fe34d5f03e2c9, scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b, user.id=6fb522b33b6a46319815df813dd33fb9) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:28.910834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a2769-9fb6-4a37-ac84-5e0dd301487c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:28.913740 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a2769-9fb6-4a37-ac84-5e0dd301487c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:28.926249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 148/295] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:28 2022] GET /identity/v3/role_assignments?user.id=6fb522b33b6a46319815df813dd33fb9&scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b&role.id=2bdabfafa05e443e842fe34d5f03e2c9 => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:28.934650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 148/296] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:28.960692 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a2769-9fb6-4a37-ac84-5e0dd301487c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:29.214314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a2769-9fb6-4a37-ac84-5e0dd301487c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:29.287855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae28469-b479-4fb6-97ef-1e1771d13b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:29.289356 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae28469-b479-4fb6-97ef-1e1771d13b41 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:29.289865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae28469-b479-4fb6-97ef-1e1771d13b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:29.290368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae28469-b479-4fb6-97ef-1e1771d13b41 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:29.312777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae28469-b479-4fb6-97ef-1e1771d13b41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-5ae28469-b479-4fb6-97ef-1e1771d13b41 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:29.326182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:29.327500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:29.328591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:29.328591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:29.328591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:29.328591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:29.328591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:29.328591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:29.328591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:29.328591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:29.328591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:29.328591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:29.328591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:29.328591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:29.328591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 149/297] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:29.337357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36859796-2df0-4790-94c5-079b269b4f60 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:29.471365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:29.548937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36859796-2df0-4790-94c5-079b269b4f60 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:29.590368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537864aa-85d8-4f6f-981c-b1705ade4e3a None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:29.591341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537864aa-85d8-4f6f-981c-b1705ade4e3a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:29.592106 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537864aa-85d8-4f6f-981c-b1705ade4e3a None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:29.595422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 150/298] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:21:29.605772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b543-dfd9-4e29-86a6-4f26ae861de1 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:29.606709 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b543-dfd9-4e29-86a6-4f26ae861de1 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:29.607570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b543-dfd9-4e29-86a6-4f26ae861de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:29.623856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8999f39-fd10-46ac-80df-9beef7874e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:29.624975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8999f39-fd10-46ac-80df-9beef7874e42 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:29.625508 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8999f39-fd10-46ac-80df-9beef7874e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:29.625977 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8999f39-fd10-46ac-80df-9beef7874e42 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:29.627348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8999f39-fd10-46ac-80df-9beef7874e42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:29.627861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8999f39-fd10-46ac-80df-9beef7874e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:29.630741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8999f39-fd10-46ac-80df-9beef7874e42 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:29.643200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-61e4b543-dfd9-4e29-86a6-4f26ae861de1 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:29.646254 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-61e4b543-dfd9-4e29-86a6-4f26ae861de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:29.650739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 149/299] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:29.663444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8999f39-fd10-46ac-80df-9beef7874e42 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:29.695474 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:29.819723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 151/300] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:29.861430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4b543-dfd9-4e29-86a6-4f26ae861de1 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:29.886335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8999f39-fd10-46ac-80df-9beef7874e42 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:29.933541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:29.957953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d2d0f-81f5-426d-a8ca-596829a202b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:29.960017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2d0f-81f5-426d-a8ca-596829a202b9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:29.960962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2d0f-81f5-426d-a8ca-596829a202b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:29.961765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2d0f-81f5-426d-a8ca-596829a202b9 None admin] PATH_INFO: `/v3/users/d6b63f5565a84cbaaf4a3cbc925772a7` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:29.986882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d2d0f-81f5-426d-a8ca-596829a202b9 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b63f5565a84cbaaf4a3cbc925772a7)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:29.988703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d2d0f-81f5-426d-a8ca-596829a202b9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:29.990791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 150/301] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:29 2022] GET /identity/v3/users/d6b63f5565a84cbaaf4a3cbc925772a7 => generated 293 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:30.001731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d2d0f-81f5-426d-a8ca-596829a202b9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:30.040037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4b543-dfd9-4e29-86a6-4f26ae861de1 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:30.077045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799992c7-41af-4c08-92f5-4abac6caafa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:30.077750 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799992c7-41af-4c08-92f5-4abac6caafa4 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:30.078287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799992c7-41af-4c08-92f5-4abac6caafa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:30.078563 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799992c7-41af-4c08-92f5-4abac6caafa4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:30.079764 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799992c7-41af-4c08-92f5-4abac6caafa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:30.081269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799992c7-41af-4c08-92f5-4abac6caafa4 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:30.089481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 152/302] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:30.089481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d78ad-ca28-4878-8a2e-c60d9e28908b None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:30.089481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d78ad-ca28-4878-8a2e-c60d9e28908b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:30.089481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d78ad-ca28-4878-8a2e-c60d9e28908b None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:30.089481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 153/303] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:30.091838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d78ad-ca28-4878-8a2e-c60d9e28908b None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:30.116572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:30.202896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d2d0f-81f5-426d-a8ca-596829a202b9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:30.211170 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d78ad-ca28-4878-8a2e-c60d9e28908b None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:30.262061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c9bfd-60ab-48ea-b1b1-a9758573c398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:30.262759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c9bfd-60ab-48ea-b1b1-a9758573c398 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:30.263046 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c9bfd-60ab-48ea-b1b1-a9758573c398 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:30.263594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c9bfd-60ab-48ea-b1b1-a9758573c398 None admin] PATH_INFO: `/v3/groups` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:30.264189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c9bfd-60ab-48ea-b1b1-a9758573c398 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:30.265755 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c9bfd-60ab-48ea-b1b1-a9758573c398 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:30.273710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d14b28-d1e0-44df-ad0b-4f3af9798273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:30.274819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d14b28-d1e0-44df-ad0b-4f3af9798273 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:30.275300 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d14b28-d1e0-44df-ad0b-4f3af9798273 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:30.275792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d14b28-d1e0-44df-ad0b-4f3af9798273 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:30.277461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d14b28-d1e0-44df-ad0b-4f3af9798273 None admin] RBAC: Adding query filter params (role.id=6a4f5f6d39ed47f4a5036b99c7386949, user.id=d6b63f5565a84cbaaf4a3cbc925772a7, scope.system=all) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:30.278026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d14b28-d1e0-44df-ad0b-4f3af9798273 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:30.281413 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d14b28-d1e0-44df-ad0b-4f3af9798273 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:30.289160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 154/304] 10.222.0.39 () {60 vars in 1134 bytes} [Tue Jun 7 08:21:30 2022] POST /identity/v3/groups => generated 229 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:21:30.291312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfe8b0-7072-45a9-94a2-61a2a5e517b9 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:30.291777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfe8b0-7072-45a9-94a2-61a2a5e517b9 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:30.292285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfe8b0-7072-45a9-94a2-61a2a5e517b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:30.303763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 151/305] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Jun 7 08:21:30 2022] GET /identity/v3/role_assignments?user.id=d6b63f5565a84cbaaf4a3cbc925772a7&scope.system=all&role.id=6a4f5f6d39ed47f4a5036b99c7386949 => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:30.322870 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-dabfe8b0-7072-45a9-94a2-61a2a5e517b9 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:30.325116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-dabfe8b0-7072-45a9-94a2-61a2a5e517b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:30.449657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 155/306] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:30.476084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d14b28-d1e0-44df-ad0b-4f3af9798273 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:30.539805 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:30.610535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d14b28-d1e0-44df-ad0b-4f3af9798273 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:30.656213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02278b-6c9c-4d11-9eaa-314598d4f5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:30.657522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02278b-6c9c-4d11-9eaa-314598d4f5bf None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:30.658055 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02278b-6c9c-4d11-9eaa-314598d4f5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:30.658558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02278b-6c9c-4d11-9eaa-314598d4f5bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:30.672517 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02278b-6c9c-4d11-9eaa-314598d4f5bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-6f02278b-6c9c-4d11-9eaa-314598d4f5bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:30.680998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:30.682103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:30.682914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:30.682914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:30.682914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:30.682914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:30.682914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:30.682914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:30.682914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:30.682914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:30.682914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:30.682914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:30.682914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:30.682914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:30.683512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 152/307] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:30.690268 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfe8b0-7072-45a9-94a2-61a2a5e517b9 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:30.704888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:30.774905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8cda3-3885-4b8b-a0c2-d4efea449ed8 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:30.775193 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8cda3-3885-4b8b-a0c2-d4efea449ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:30.775560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8cda3-3885-4b8b-a0c2-d4efea449ed8 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:30.776996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 153/308] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:30.783072 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6a661-3714-4b74-9885-76e9aeb73e86 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:30.783336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6a661-3714-4b74-9885-76e9aeb73e86 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:30.783643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6a661-3714-4b74-9885-76e9aeb73e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:30.801833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-33d6a661-3714-4b74-9885-76e9aeb73e86 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:30.803047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-33d6a661-3714-4b74-9885-76e9aeb73e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:30.841023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfe8b0-7072-45a9-94a2-61a2a5e517b9 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:30.879330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 154/309] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:30.911199 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fcd64-8100-4480-a18e-b26d7b95c5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:30.913157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fcd64-8100-4480-a18e-b26d7b95c5bf None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:30.913808 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d6a661-3714-4b74-9885-76e9aeb73e86 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:30.914259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fcd64-8100-4480-a18e-b26d7b95c5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:30.914988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fcd64-8100-4480-a18e-b26d7b95c5bf None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:30.916858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fcd64-8100-4480-a18e-b26d7b95c5bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:30.918064 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fcd64-8100-4480-a18e-b26d7b95c5bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:30.921058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fcd64-8100-4480-a18e-b26d7b95c5bf None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:30.938406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 156/310] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:30.944553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fcd64-8100-4480-a18e-b26d7b95c5bf None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:30.958631 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:31.033738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d6a661-3714-4b74-9885-76e9aeb73e86 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:31.071670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23ce4e-b9d9-4110-a092-529036cde102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:31.072872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23ce4e-b9d9-4110-a092-529036cde102 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:31.073394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23ce4e-b9d9-4110-a092-529036cde102 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:31.073893 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23ce4e-b9d9-4110-a092-529036cde102 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:31.086246 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23ce4e-b9d9-4110-a092-529036cde102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:31.088813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-ba23ce4e-b9d9-4110-a092-529036cde102 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:31.095919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:31.096855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:31.097691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:31.097691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:31.097691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:31.097691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:31.097691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:31.097691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:31.097691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:31.097691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:31.097691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:31.097691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:31.097691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:31.097691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:31.100259 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 155/311] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:31.106066 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23ce4e-b9d9-4110-a092-529036cde102 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:31.137893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fcd64-8100-4480-a18e-b26d7b95c5bf None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:31.210478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768232e-31ce-4e88-a8b9-a4dc181334dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:31.212089 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768232e-31ce-4e88-a8b9-a4dc181334dc None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:31.212587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768232e-31ce-4e88-a8b9-a4dc181334dc None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:31.213427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768232e-31ce-4e88-a8b9-a4dc181334dc None admin] PATH_INFO: `/v3/users/7c77983a5fc443ed916b8af81f638df6` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:31.215556 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768232e-31ce-4e88-a8b9-a4dc181334dc None admin] RBAC: Authorizing `identity:get_user(user_id=7c77983a5fc443ed916b8af81f638df6)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:31.216541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:31.218434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768232e-31ce-4e88-a8b9-a4dc181334dc None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:31.222514 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 157/312] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:30 2022] GET /identity/v3/users/7c77983a5fc443ed916b8af81f638df6 => generated 297 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:31.228028 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768232e-31ce-4e88-a8b9-a4dc181334dc None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:31.239587 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23ce4e-b9d9-4110-a092-529036cde102 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:31.280602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e696b3-8582-4a65-abfa-ce51d4cd6f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:31.281352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e696b3-8582-4a65-abfa-ce51d4cd6f0d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:31.281683 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e696b3-8582-4a65-abfa-ce51d4cd6f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:31.281969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e696b3-8582-4a65-abfa-ce51d4cd6f0d None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:31.282819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e696b3-8582-4a65-abfa-ce51d4cd6f0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:31.283043 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e696b3-8582-4a65-abfa-ce51d4cd6f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:31.284448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e696b3-8582-4a65-abfa-ce51d4cd6f0d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:31.295064 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 156/313] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:31.300716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e696b3-8582-4a65-abfa-ce51d4cd6f0d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:31.413886 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e696b3-8582-4a65-abfa-ce51d4cd6f0d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:31.443747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768232e-31ce-4e88-a8b9-a4dc181334dc None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:31.451298 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb9863-590d-4f7b-9ec8-40a60bc4b7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:31.452299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb9863-590d-4f7b-9ec8-40a60bc4b7bd None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:31.452588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb9863-590d-4f7b-9ec8-40a60bc4b7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:31.452917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb9863-590d-4f7b-9ec8-40a60bc4b7bd None admin] PATH_INFO: `/v3/users/55a15f45d2534b4e80c552fe0b8133cd` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:31.454487 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb9863-590d-4f7b-9ec8-40a60bc4b7bd None admin] RBAC: Authorizing `identity:get_user(user_id=55a15f45d2534b4e80c552fe0b8133cd)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:31.455996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb9863-590d-4f7b-9ec8-40a60bc4b7bd None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:31.457736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 157/314] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:31 2022] GET /identity/v3/users/55a15f45d2534b4e80c552fe0b8133cd => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:31.463543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb9863-590d-4f7b-9ec8-40a60bc4b7bd None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:31.491488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:31.520144 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cf584-310f-46ca-8460-55fb4c639244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:31.521845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cf584-310f-46ca-8460-55fb4c639244 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:31.522608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cf584-310f-46ca-8460-55fb4c639244 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:31.523347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cf584-310f-46ca-8460-55fb4c639244 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:31.546602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:31.574518 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb9863-590d-4f7b-9ec8-40a60bc4b7bd None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:31.612411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9c80c-f1c4-42d8-9a3a-dddab518b614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:31.613448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9c80c-f1c4-42d8-9a3a-dddab518b614 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:31.613728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9c80c-f1c4-42d8-9a3a-dddab518b614 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:31.613954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9c80c-f1c4-42d8-9a3a-dddab518b614 None admin] PATH_INFO: `/v3/system/users/55a15f45d2534b4e80c552fe0b8133cd/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:31.615513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9c80c-f1c4-42d8-9a3a-dddab518b614 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=55a15f45d2534b4e80c552fe0b8133cd, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:31.616963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9c80c-f1c4-42d8-9a3a-dddab518b614 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:31.633327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 158/315] 10.222.0.39 () {58 vars in 1334 bytes} [Tue Jun 7 08:21:31 2022] PUT /identity/v3/system/users/55a15f45d2534b4e80c552fe0b8133cd/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:31.674515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cf584-310f-46ca-8460-55fb4c639244 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:31.678132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cf584-310f-46ca-8460-55fb4c639244 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:31.883876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 158/316] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:21:31.891241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9c80c-f1c4-42d8-9a3a-dddab518b614 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:32.010451 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:32.036290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9c80c-f1c4-42d8-9a3a-dddab518b614 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:32.077940 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4a1a2-de91-4967-8145-e2a8dc0ead8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:32.079073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4a1a2-de91-4967-8145-e2a8dc0ead8c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:32.079357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4a1a2-de91-4967-8145-e2a8dc0ead8c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:32.079640 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4a1a2-de91-4967-8145-e2a8dc0ead8c None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:32.080945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce4a1a2-de91-4967-8145-e2a8dc0ead8c None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:32.082458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce4a1a2-de91-4967-8145-e2a8dc0ead8c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:32.084552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 159/317] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:31 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:32.094974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cf584-310f-46ca-8460-55fb4c639244 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:32.168315 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:32.245206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cf584-310f-46ca-8460-55fb4c639244 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:32.287365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9651b-b405-429f-bc43-fa2947ff5f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:32.288050 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9651b-b405-429f-bc43-fa2947ff5f07 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:32.288336 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9651b-b405-429f-bc43-fa2947ff5f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:32.288615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9651b-b405-429f-bc43-fa2947ff5f07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:32.289502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9651b-b405-429f-bc43-fa2947ff5f07 None admin] RBAC: Adding query filter params (role.id=6a4f5f6d39ed47f4a5036b99c7386949, scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b, user.id=7c77983a5fc443ed916b8af81f638df6) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:32.289821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9651b-b405-429f-bc43-fa2947ff5f07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:32.291338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9651b-b405-429f-bc43-fa2947ff5f07 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:32.299372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 159/318] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:32 2022] GET /identity/v3/role_assignments?user.id=7c77983a5fc443ed916b8af81f638df6&scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b&role.id=6a4f5f6d39ed47f4a5036b99c7386949 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:33.195784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e919cd-9b12-4beb-95ef-98337d015cfa None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:33.196288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e919cd-9b12-4beb-95ef-98337d015cfa None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:33.196288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e919cd-9b12-4beb-95ef-98337d015cfa None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:33.197730 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 160/319] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:33.204266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767f036-5693-4893-bdc6-c77bd691fa2b None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:33.204836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767f036-5693-4893-bdc6-c77bd691fa2b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:33.204836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767f036-5693-4893-bdc6-c77bd691fa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:33.223526 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e767f036-5693-4893-bdc6-c77bd691fa2b None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:33.224752 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e767f036-5693-4893-bdc6-c77bd691fa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:33.316418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 160/320] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:33.343752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e919cd-9b12-4beb-95ef-98337d015cfa None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:33.416069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265b358-e45f-4042-abee-35ed75faf349 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:33.416467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265b358-e45f-4042-abee-35ed75faf349 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:33.416807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265b358-e45f-4042-abee-35ed75faf349 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:33.418533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 161/321] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:21:33.424373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd651057-07d4-48c8-9d55-649fc1384b82 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:33.424765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd651057-07d4-48c8-9d55-649fc1384b82 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:33.424919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd651057-07d4-48c8-9d55-649fc1384b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:33.442581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-cd651057-07d4-48c8-9d55-649fc1384b82 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:33.443780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-cd651057-07d4-48c8-9d55-649fc1384b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:33.467426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e919cd-9b12-4beb-95ef-98337d015cfa None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:33.475519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:33.509266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27316f39-a721-43b2-8aaa-6648ef551833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:33.510245 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27316f39-a721-43b2-8aaa-6648ef551833 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:33.510686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27316f39-a721-43b2-8aaa-6648ef551833 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:33.511146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27316f39-a721-43b2-8aaa-6648ef551833 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:33.512794 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27316f39-a721-43b2-8aaa-6648ef551833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:33.515419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27316f39-a721-43b2-8aaa-6648ef551833 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:33.517921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 161/322] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:33.524534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27316f39-a721-43b2-8aaa-6648ef551833 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:33.529844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 162/323] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:33.552341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd651057-07d4-48c8-9d55-649fc1384b82 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:33.553523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:33.643958 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27316f39-a721-43b2-8aaa-6648ef551833 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:33.660551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd651057-07d4-48c8-9d55-649fc1384b82 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:33.680603 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a72a4-ea6d-431b-bcaf-c0e27e339548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:33.681294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a72a4-ea6d-431b-bcaf-c0e27e339548 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:33.681580 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a72a4-ea6d-431b-bcaf-c0e27e339548 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:33.681856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a72a4-ea6d-431b-bcaf-c0e27e339548 None admin] PATH_INFO: `/v3/groups` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:33.682548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a72a4-ea6d-431b-bcaf-c0e27e339548 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:33.684023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a72a4-ea6d-431b-bcaf-c0e27e339548 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:33.701600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 162/324] 10.222.0.39 () {60 vars in 1134 bytes} [Tue Jun 7 08:21:33 2022] POST /identity/v3/groups => generated 226 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:21:33.704522 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066f0d1-add2-4248-b779-145d9e3cc5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:33.705433 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066f0d1-add2-4248-b779-145d9e3cc5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:33.705784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066f0d1-add2-4248-b779-145d9e3cc5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:33.706148 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066f0d1-add2-4248-b779-145d9e3cc5e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:33.724796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066f0d1-add2-4248-b779-145d9e3cc5e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-f066f0d1-add2-4248-b779-145d9e3cc5e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:33.733992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:33.735021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:33.735826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:33.735826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:33.735826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:33.735826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:33.735826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:33.735826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:33.735826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:33.735826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:33.735826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:33.735826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:33.735826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:33.735826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:33.735826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 163/325] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:33.742858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a72a4-ea6d-431b-bcaf-c0e27e339548 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:33.851154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:33.879225 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a72a4-ea6d-431b-bcaf-c0e27e339548 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:33.936187 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaaab6-1f01-4ba4-a6c1-98b73e6af600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:33.937398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaaab6-1f01-4ba4-a6c1-98b73e6af600 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:33.937840 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaaab6-1f01-4ba4-a6c1-98b73e6af600 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:33.938115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaaab6-1f01-4ba4-a6c1-98b73e6af600 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:33.938883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaaab6-1f01-4ba4-a6c1-98b73e6af600 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:33.939187 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaaab6-1f01-4ba4-a6c1-98b73e6af600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:33.940584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaaab6-1f01-4ba4-a6c1-98b73e6af600 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:33.950768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 163/326] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:33.957878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066f0d1-add2-4248-b779-145d9e3cc5e9 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:33.995898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:34.094743 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066f0d1-add2-4248-b779-145d9e3cc5e9 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:34.145538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c69d0-3a93-47f5-b7c7-9ad1f144673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:34.146448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c69d0-3a93-47f5-b7c7-9ad1f144673c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:34.146753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c69d0-3a93-47f5-b7c7-9ad1f144673c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:34.147005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c69d0-3a93-47f5-b7c7-9ad1f144673c None admin] PATH_INFO: `/v3/users/d6b63f5565a84cbaaf4a3cbc925772a7` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:34.148379 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c69d0-3a93-47f5-b7c7-9ad1f144673c None admin] RBAC: Authorizing `identity:get_user(user_id=d6b63f5565a84cbaaf4a3cbc925772a7)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:34.149939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c69d0-3a93-47f5-b7c7-9ad1f144673c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:34.152180 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 164/327] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:33 2022] GET /identity/v3/users/d6b63f5565a84cbaaf4a3cbc925772a7 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:34.158834 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaaab6-1f01-4ba4-a6c1-98b73e6af600 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:34.303527 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaaab6-1f01-4ba4-a6c1-98b73e6af600 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:34.348284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5153d-3e75-4b11-bc48-c30f28f1dd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:34.349677 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5153d-3e75-4b11-bc48-c30f28f1dd24 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:34.350195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5153d-3e75-4b11-bc48-c30f28f1dd24 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:34.350686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5153d-3e75-4b11-bc48-c30f28f1dd24 None admin] PATH_INFO: `/v3/system/users/d6b63f5565a84cbaaf4a3cbc925772a7/roles/6a4f5f6d39ed47f4a5036b99c7386949` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:34.355005 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5153d-3e75-4b11-bc48-c30f28f1dd24 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d6b63f5565a84cbaaf4a3cbc925772a7, role_id=6a4f5f6d39ed47f4a5036b99c7386949)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:34.357295 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5153d-3e75-4b11-bc48-c30f28f1dd24 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:34.367627 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 164/328] 10.222.0.39 () {58 vars in 1334 bytes} [Tue Jun 7 08:21:34 2022] PUT /identity/v3/system/users/d6b63f5565a84cbaaf4a3cbc925772a7/roles/6a4f5f6d39ed47f4a5036b99c7386949 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:34.413940 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4b131-6792-4dc1-8c77-ea245f9c3927 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:34.414612 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4b131-6792-4dc1-8c77-ea245f9c3927 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:34.414612 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4b131-6792-4dc1-8c77-ea245f9c3927 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:34.416098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 165/329] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:34.423643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44974fd3-cd65-4475-8698-7d1e28e7c3c9 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:34.424124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44974fd3-cd65-4475-8698-7d1e28e7c3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:34.424595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44974fd3-cd65-4475-8698-7d1e28e7c3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:34.456951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-44974fd3-cd65-4475-8698-7d1e28e7c3c9 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:34.459366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-44974fd3-cd65-4475-8698-7d1e28e7c3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:34.464977 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:34.640159 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 165/330] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:34.667002 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4b131-6792-4dc1-8c77-ea245f9c3927 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:34.799223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:34.817159 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4b131-6792-4dc1-8c77-ea245f9c3927 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:34.866769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e875ae0-f0a5-40a1-8a2a-ac2b1f5c7108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:34.867570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e875ae0-f0a5-40a1-8a2a-ac2b1f5c7108 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:34.867855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e875ae0-f0a5-40a1-8a2a-ac2b1f5c7108 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:34.868126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e875ae0-f0a5-40a1-8a2a-ac2b1f5c7108 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:34.882349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e875ae0-f0a5-40a1-8a2a-ac2b1f5c7108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-9e875ae0-f0a5-40a1-8a2a-ac2b1f5c7108 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:34.895668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:34.896594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:34.897408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:34.897408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:34.897408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:34.897408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:34.897408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:34.897408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:34.897408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:34.897408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:34.897408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:34.897408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:34.897408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:34.897408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:34.897408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 166/331] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:34.910550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44974fd3-cd65-4475-8698-7d1e28e7c3c9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:35.119630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44974fd3-cd65-4475-8698-7d1e28e7c3c9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:35.183210 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2130cc1-2415-412d-8bb0-3f0c60ab9024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:35.184251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2130cc1-2415-412d-8bb0-3f0c60ab9024 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:35.184752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2130cc1-2415-412d-8bb0-3f0c60ab9024 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:35.185806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2130cc1-2415-412d-8bb0-3f0c60ab9024 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:35.186763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2130cc1-2415-412d-8bb0-3f0c60ab9024 None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:35.187273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2130cc1-2415-412d-8bb0-3f0c60ab9024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:35.188888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2130cc1-2415-412d-8bb0-3f0c60ab9024 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:35.202768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 166/332] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:35.211441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e875ae0-f0a5-40a1-8a2a-ac2b1f5c7108 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:35.438631 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e875ae0-f0a5-40a1-8a2a-ac2b1f5c7108 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:35.512225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a22e67-e929-472e-9a14-74ef7db22da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:35.514488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a22e67-e929-472e-9a14-74ef7db22da1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:35.514985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a22e67-e929-472e-9a14-74ef7db22da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:35.515478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a22e67-e929-472e-9a14-74ef7db22da1 None admin] PATH_INFO: `/v3/users/55a15f45d2534b4e80c552fe0b8133cd` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:35.522331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a22e67-e929-472e-9a14-74ef7db22da1 None admin] RBAC: Authorizing `identity:get_user(user_id=55a15f45d2534b4e80c552fe0b8133cd)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:35.525041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a22e67-e929-472e-9a14-74ef7db22da1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:35.528489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 167/333] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:35 2022] GET /identity/v3/users/55a15f45d2534b4e80c552fe0b8133cd => generated 293 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:35.537628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2130cc1-2415-412d-8bb0-3f0c60ab9024 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:35.541239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:35.715690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2130cc1-2415-412d-8bb0-3f0c60ab9024 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:35.765926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5b2ac-7984-4332-85ad-32e7527335ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:35.766981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5b2ac-7984-4332-85ad-32e7527335ac None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:35.767470 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5b2ac-7984-4332-85ad-32e7527335ac None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:35.767944 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5b2ac-7984-4332-85ad-32e7527335ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:35.768997 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5b2ac-7984-4332-85ad-32e7527335ac None admin] RBAC: Adding query filter params (role.id=2bdabfafa05e443e842fe34d5f03e2c9, user.id=55a15f45d2534b4e80c552fe0b8133cd, scope.system=all) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:35.769545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5b2ac-7984-4332-85ad-32e7527335ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:35.771209 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5b2ac-7984-4332-85ad-32e7527335ac None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:35.789613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 167/334] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Jun 7 08:21:35 2022] GET /identity/v3/role_assignments?user.id=55a15f45d2534b4e80c552fe0b8133cd&scope.system=all&role.id=2bdabfafa05e443e842fe34d5f03e2c9 => generated 510 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:37.096299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466eecd-8a05-4000-bc69-fda945acdd8a None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:37.097358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466eecd-8a05-4000-bc69-fda945acdd8a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:37.097893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466eecd-8a05-4000-bc69-fda945acdd8a None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:37.099722 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 168/335] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:37.105534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08026273-7360-4658-a4c2-4e971c2f6857 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:37.106422 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08026273-7360-4658-a4c2-4e971c2f6857 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:37.107488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08026273-7360-4658-a4c2-4e971c2f6857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:37.134473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-08026273-7360-4658-a4c2-4e971c2f6857 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:37.136015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-08026273-7360-4658-a4c2-4e971c2f6857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:37.224253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:37.237105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 168/336] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:37.262325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466eecd-8a05-4000-bc69-fda945acdd8a None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:37.291648 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:37.403219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466eecd-8a05-4000-bc69-fda945acdd8a None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:37.447682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbd814-7308-448c-aad1-c12d0300bae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:37.448501 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbd814-7308-448c-aad1-c12d0300bae3 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:37.448786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbd814-7308-448c-aad1-c12d0300bae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:37.449060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbd814-7308-448c-aad1-c12d0300bae3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:37.464756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbd814-7308-448c-aad1-c12d0300bae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-37bbd814-7308-448c-aad1-c12d0300bae3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:37.473125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:37.474040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:37.474901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:37.474901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:37.474901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:37.474901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:37.474901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:37.474901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:37.474901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:37.474901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:37.474901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:37.474901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:37.474901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:37.474901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:37.474901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 169/337] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:21:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:37.482453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08026273-7360-4658-a4c2-4e971c2f6857 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:37.627084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08026273-7360-4658-a4c2-4e971c2f6857 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:37.680477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367b2d9-2d21-4980-ad82-a3977bb85772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:37.681586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367b2d9-2d21-4980-ad82-a3977bb85772 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:37.682071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367b2d9-2d21-4980-ad82-a3977bb85772 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:37.682554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367b2d9-2d21-4980-ad82-a3977bb85772 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:37.683519 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367b2d9-2d21-4980-ad82-a3977bb85772 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:37.684020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367b2d9-2d21-4980-ad82-a3977bb85772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:37.686340 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367b2d9-2d21-4980-ad82-a3977bb85772 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:37.703555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 169/338] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:37.710479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbd814-7308-448c-aad1-c12d0300bae3 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:37.732696 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee148079-25e6-4227-87f7-9be4db54e266 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:37.733409 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee148079-25e6-4227-87f7-9be4db54e266 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:37.733912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee148079-25e6-4227-87f7-9be4db54e266 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:37.735661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 170/339] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:21:37.741932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1651e9d-ce4e-493d-af05-63f93f0ac748 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:37.742545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1651e9d-ce4e-493d-af05-63f93f0ac748 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:37.743031 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1651e9d-ce4e-493d-af05-63f93f0ac748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:37.775065 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b1651e9d-ce4e-493d-af05-63f93f0ac748 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:37.776577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b1651e9d-ce4e-493d-af05-63f93f0ac748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:37.858311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:37.891632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:37.904452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 171/340] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:37.912836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbd814-7308-448c-aad1-c12d0300bae3 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:37.959931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fff9d-77ca-4fae-9f08-300470319192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:37.960620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fff9d-77ca-4fae-9f08-300470319192 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:37.960913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fff9d-77ca-4fae-9f08-300470319192 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:37.961239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fff9d-77ca-4fae-9f08-300470319192 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:37.965534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1651e9d-ce4e-493d-af05-63f93f0ac748 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:38.148292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fff9d-77ca-4fae-9f08-300470319192 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:38.150077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fff9d-77ca-4fae-9f08-300470319192 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:38.292624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:38.302758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1651e9d-ce4e-493d-af05-63f93f0ac748 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:38.319579 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 170/341] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:38.325188 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fff9d-77ca-4fae-9f08-300470319192 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:38.390018 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872d610-e21e-416b-8fc5-27959d3b0032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:38.390903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872d610-e21e-416b-8fc5-27959d3b0032 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:38.391202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872d610-e21e-416b-8fc5-27959d3b0032 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:38.391503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872d610-e21e-416b-8fc5-27959d3b0032 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:38.419156 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872d610-e21e-416b-8fc5-27959d3b0032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-5872d610-e21e-416b-8fc5-27959d3b0032 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:38.433253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:38.434203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:38.435612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:38.435612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:38.435612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:38.435612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:38.435612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:38.435612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:38.435612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:38.435612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:38.435612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:38.435612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:38.435612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:38.435612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:38.439939 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 172/342] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 470 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:38.441275 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872d610-e21e-416b-8fc5-27959d3b0032 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:38.520130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:38.539160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fff9d-77ca-4fae-9f08-300470319192 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:38.604641 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ce8f8-b151-430f-982a-778118bee0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:38.605861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ce8f8-b151-430f-982a-778118bee0bb None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:38.609947 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ce8f8-b151-430f-982a-778118bee0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:38.610328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ce8f8-b151-430f-982a-778118bee0bb None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:38.611882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ce8f8-b151-430f-982a-778118bee0bb None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:38.613443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ce8f8-b151-430f-982a-778118bee0bb None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:38.619031 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 171/343] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:38 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:38.620530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12b989-5115-45e4-a732-f229351e9c29 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:38.620816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12b989-5115-45e4-a732-f229351e9c29 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:38.621128 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12b989-5115-45e4-a732-f229351e9c29 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:38.622896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 172/344] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:38.625542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12b989-5115-45e4-a732-f229351e9c29 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:38.638392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872d610-e21e-416b-8fc5-27959d3b0032 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:38.708402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcd7be-834f-4e2d-8cea-7e5ba62b047c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:38.709619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcd7be-834f-4e2d-8cea-7e5ba62b047c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:38.710131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcd7be-834f-4e2d-8cea-7e5ba62b047c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:38.710622 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcd7be-834f-4e2d-8cea-7e5ba62b047c None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:38.711604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcd7be-834f-4e2d-8cea-7e5ba62b047c None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:38.712228 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcd7be-834f-4e2d-8cea-7e5ba62b047c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:38.713981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcd7be-834f-4e2d-8cea-7e5ba62b047c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:38.716918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:38.726939 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 173/345] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:38.728711 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6406ec-7ce0-4e51-b2c4-1c1835075644 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:38.729286 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6406ec-7ce0-4e51-b2c4-1c1835075644 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:38.729802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6406ec-7ce0-4e51-b2c4-1c1835075644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:38.758413 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4f6406ec-7ce0-4e51-b2c4-1c1835075644 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:38.760160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4f6406ec-7ce0-4e51-b2c4-1c1835075644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:38.808897 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12b989-5115-45e4-a732-f229351e9c29 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:38.860293 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52692604-a9c9-4b87-b294-daea9ea9984c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:38.861543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52692604-a9c9-4b87-b294-daea9ea9984c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:38.862030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52692604-a9c9-4b87-b294-daea9ea9984c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:38.862535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52692604-a9c9-4b87-b294-daea9ea9984c None admin] PATH_INFO: `/v3/groups/0a07d620279f4fcab3723e3460d7d2bc` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:38.866092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 174/346] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:38.867566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0eb7c-b793-4d34-adc5-564a13890615 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:38.867864 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0eb7c-b793-4d34-adc5-564a13890615 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:38.868147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0eb7c-b793-4d34-adc5-564a13890615 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:38.870870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 175/347] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:38.875972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0eb7c-b793-4d34-adc5-564a13890615 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:38.876942 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52692604-a9c9-4b87-b294-daea9ea9984c None admin] RBAC: Authorizing `identity:get_group(group_id=0a07d620279f4fcab3723e3460d7d2bc)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:38.878719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52692604-a9c9-4b87-b294-daea9ea9984c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:38.880962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 173/348] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:38 2022] GET /identity/v3/groups/0a07d620279f4fcab3723e3460d7d2bc => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:38.882458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c155e5-aecf-4d78-9e63-6550fef4a63c None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:38.882968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c155e5-aecf-4d78-9e63-6550fef4a63c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:38.883465 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c155e5-aecf-4d78-9e63-6550fef4a63c None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:38.885654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 174/349] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:38.887012 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbad43-e2b2-4a8e-983a-793a659f5065 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:38.887500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbad43-e2b2-4a8e-983a-793a659f5065 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:38.897424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbad43-e2b2-4a8e-983a-793a659f5065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:38.917944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-8dfbad43-e2b2-4a8e-983a-793a659f5065 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:38.919965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8dfbad43-e2b2-4a8e-983a-793a659f5065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:38.939466 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:39.014675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 175/350] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:39.015893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbad43-e2b2-4a8e-983a-793a659f5065 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:39.017898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0eb7c-b793-4d34-adc5-564a13890615 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:39.067257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257e0ad-0d4b-4b1c-a3bb-53313c219873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:39.067925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257e0ad-0d4b-4b1c-a3bb-53313c219873 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.068203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257e0ad-0d4b-4b1c-a3bb-53313c219873 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.068472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257e0ad-0d4b-4b1c-a3bb-53313c219873 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.078582 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:39.161531 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbad43-e2b2-4a8e-983a-793a659f5065 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:39.168054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257e0ad-0d4b-4b1c-a3bb-53313c219873 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:39.169972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257e0ad-0d4b-4b1c-a3bb-53313c219873 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:39.217054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:39.218020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80878f-6991-4f40-a5ef-3e513e04d89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:39.218539 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80878f-6991-4f40-a5ef-3e513e04d89f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.218828 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80878f-6991-4f40-a5ef-3e513e04d89f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.219250 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80878f-6991-4f40-a5ef-3e513e04d89f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.220179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80878f-6991-4f40-a5ef-3e513e04d89f None admin] RBAC: Adding query filter params (group.id=0a07d620279f4fcab3723e3460d7d2bc, role.id=5addc8ffe20f47749792ac1394f6235c, scope.project.id=d00b803196664866a4fd2eb970b1be0b) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:39.220499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80878f-6991-4f40-a5ef-3e513e04d89f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:39.221993 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80878f-6991-4f40-a5ef-3e513e04d89f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:39.230306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 176/351] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jun 7 08:21:39 2022] GET /identity/v3/role_assignments?group.id=0a07d620279f4fcab3723e3460d7d2bc&scope.project.id=d00b803196664866a4fd2eb970b1be0b&role.id=5addc8ffe20f47749792ac1394f6235c => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:39.232153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80878f-6991-4f40-a5ef-3e513e04d89f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:39.324097 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 176/352] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:39.327342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cf57d-8f91-4c71-8820-7f9e2fab09d0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.327342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cf57d-8f91-4c71-8820-7f9e2fab09d0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.327342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cf57d-8f91-4c71-8820-7f9e2fab09d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.346033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7b5cf57d-8f91-4c71-8820-7f9e2fab09d0 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:39.347719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7b5cf57d-8f91-4c71-8820-7f9e2fab09d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:39.378030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80878f-6991-4f40-a5ef-3e513e04d89f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:39.386390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:39.419867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f2131-30fb-432d-8321-10a02444cfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:39.421359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f2131-30fb-432d-8321-10a02444cfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.421359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f2131-30fb-432d-8321-10a02444cfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.421720 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f2131-30fb-432d-8321-10a02444cfb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.436917 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f2131-30fb-432d-8321-10a02444cfb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-064f2131-30fb-432d-8321-10a02444cfb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:39.445479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:39.446472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:39.447284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:39.447284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:39.447284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:39.447284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:39.447284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:39.447284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:39.447284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:39.447284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:39.447284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:39.447284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:39.447284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:21:39.447284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:39.447284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 177/353] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:39.447943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f2131-30fb-432d-8321-10a02444cfb3 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:39.450477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 177/354] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:39.453277 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5cf57d-8f91-4c71-8820-7f9e2fab09d0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:39.524681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:39.566622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f2131-30fb-432d-8321-10a02444cfb3 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:39.568997 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5cf57d-8f91-4c71-8820-7f9e2fab09d0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:39.608239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4435057-ac11-418b-8f07-539319f52b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:39.609214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4435057-ac11-418b-8f07-539319f52b5b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.609622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4435057-ac11-418b-8f07-539319f52b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.609909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4435057-ac11-418b-8f07-539319f52b5b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.614809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d995ab3-a849-49c8-a291-e16082439202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:39.615957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d995ab3-a849-49c8-a291-e16082439202 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.616439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d995ab3-a849-49c8-a291-e16082439202 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.616987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d995ab3-a849-49c8-a291-e16082439202 None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.618562 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d995ab3-a849-49c8-a291-e16082439202 None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:39.620184 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d995ab3-a849-49c8-a291-e16082439202 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:39.622171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 178/355] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:39 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:39.623706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d995ab3-a849-49c8-a291-e16082439202 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:39.630502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4435057-ac11-418b-8f07-539319f52b5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-a4435057-ac11-418b-8f07-539319f52b5b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:39.642171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:39.643081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:39.643877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:39.643877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:39.643877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:39.643877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:39.643877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:39.643877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:39.643877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:39.643877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:39.643877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:39.643877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:39.643877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:39.643877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:39.643877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 178/356] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:21:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:39.644535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4435057-ac11-418b-8f07-539319f52b5b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:39.676347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:39.734145 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d995ab3-a849-49c8-a291-e16082439202 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:39.761691 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4435057-ac11-418b-8f07-539319f52b5b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:39.772788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b1f47-7412-4e66-a2ac-adb24b21f1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:39.773806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b1f47-7412-4e66-a2ac-adb24b21f1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.774267 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b1f47-7412-4e66-a2ac-adb24b21f1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.774697 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b1f47-7412-4e66-a2ac-adb24b21f1a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.775632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b1f47-7412-4e66-a2ac-adb24b21f1a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:39.776102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b1f47-7412-4e66-a2ac-adb24b21f1a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:39.777726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b1f47-7412-4e66-a2ac-adb24b21f1a9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:39.789480 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:39.793498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 179/357] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:39.794806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b1f47-7412-4e66-a2ac-adb24b21f1a9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:39.814549 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd59da3-4183-4500-a6be-25ed9b9f17a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:39.815364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd59da3-4183-4500-a6be-25ed9b9f17a3 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.815652 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd59da3-4183-4500-a6be-25ed9b9f17a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.816111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd59da3-4183-4500-a6be-25ed9b9f17a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.835153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd59da3-4183-4500-a6be-25ed9b9f17a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-ecd59da3-4183-4500-a6be-25ed9b9f17a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:39.844452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:39.845420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:39.846210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:39.846210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:39.846210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:39.846210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:39.846210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:39.846210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:39.846210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:39.846210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:39.846210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:39.846210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:39.846210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:39.846210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:39.846210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 179/358] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:39.846950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd59da3-4183-4500-a6be-25ed9b9f17a3 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:39.941992 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b1f47-7412-4e66-a2ac-adb24b21f1a9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:39.980254 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:39.987643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128b0f5-3205-4062-8e97-a486b02d4276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:39.989266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128b0f5-3205-4062-8e97-a486b02d4276 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.989556 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128b0f5-3205-4062-8e97-a486b02d4276 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.990018 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128b0f5-3205-4062-8e97-a486b02d4276 None admin] PATH_INFO: `/v3/groups/0a07d620279f4fcab3723e3460d7d2bc` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.991569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128b0f5-3205-4062-8e97-a486b02d4276 None admin] RBAC: Authorizing `identity:get_group(group_id=0a07d620279f4fcab3723e3460d7d2bc)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:39.993884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128b0f5-3205-4062-8e97-a486b02d4276 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:39.996551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 180/359] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:39 2022] GET /identity/v3/groups/0a07d620279f4fcab3723e3460d7d2bc => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:39.998073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128b0f5-3205-4062-8e97-a486b02d4276 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:40.000285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd59da3-4183-4500-a6be-25ed9b9f17a3 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:40.049875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8ad04-3544-4119-b95b-30316152777b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:40.051248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8ad04-3544-4119-b95b-30316152777b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:40.051564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8ad04-3544-4119-b95b-30316152777b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:40.051845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8ad04-3544-4119-b95b-30316152777b None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:40.052656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8ad04-3544-4119-b95b-30316152777b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:40.053498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8ad04-3544-4119-b95b-30316152777b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:40.055238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8ad04-3544-4119-b95b-30316152777b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:40.064661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 180/360] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:40.065551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8ad04-3544-4119-b95b-30316152777b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:40.084646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:40.140659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128b0f5-3205-4062-8e97-a486b02d4276 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:40.188303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b417c9d-15f8-4c9f-b353-5f70d1325f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:40.190000 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b417c9d-15f8-4c9f-b353-5f70d1325f31 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:40.190000 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b417c9d-15f8-4c9f-b353-5f70d1325f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:40.190000 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b417c9d-15f8-4c9f-b353-5f70d1325f31 None admin] PATH_INFO: `/v3/users/d6b63f5565a84cbaaf4a3cbc925772a7` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:40.191143 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b417c9d-15f8-4c9f-b353-5f70d1325f31 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b63f5565a84cbaaf4a3cbc925772a7)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:40.192582 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b417c9d-15f8-4c9f-b353-5f70d1325f31 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:40.194314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 181/361] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:39 2022] GET /identity/v3/users/d6b63f5565a84cbaaf4a3cbc925772a7 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:40.195542 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2278b-6b93-4833-9ff6-57a374069b9d None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:40.195829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2278b-6b93-4833-9ff6-57a374069b9d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:40.196106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2278b-6b93-4833-9ff6-57a374069b9d None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:40.197541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 182/362] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:40.198129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2278b-6b93-4833-9ff6-57a374069b9d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:40.211410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8ad04-3544-4119-b95b-30316152777b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:40.253466 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cdb09-07eb-46f2-a7bc-11fb569841c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:40.254365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cdb09-07eb-46f2-a7bc-11fb569841c2 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:40.254810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cdb09-07eb-46f2-a7bc-11fb569841c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:40.255271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cdb09-07eb-46f2-a7bc-11fb569841c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:40.256513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1cdb09-07eb-46f2-a7bc-11fb569841c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:40.257012 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1cdb09-07eb-46f2-a7bc-11fb569841c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:40.259544 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1cdb09-07eb-46f2-a7bc-11fb569841c2 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:40.272487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 181/363] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:40.273634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cdb09-07eb-46f2-a7bc-11fb569841c2 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:40.289853 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:40.356817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2278b-6b93-4833-9ff6-57a374069b9d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:40.400545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:40.406738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16263583-a65f-4bfa-b07f-e41679b673bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:40.407406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16263583-a65f-4bfa-b07f-e41679b673bf None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:40.407695 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16263583-a65f-4bfa-b07f-e41679b673bf None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:40.407964 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16263583-a65f-4bfa-b07f-e41679b673bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:40.408812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16263583-a65f-4bfa-b07f-e41679b673bf None admin] RBAC: Adding query filter params (group.id=0a07d620279f4fcab3723e3460d7d2bc, role.id=2bdabfafa05e443e842fe34d5f03e2c9, scope.project.id=d00b803196664866a4fd2eb970b1be0b) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:40.409148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16263583-a65f-4bfa-b07f-e41679b673bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:40.409266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cdb09-07eb-46f2-a7bc-11fb569841c2 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:40.410591 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16263583-a65f-4bfa-b07f-e41679b673bf None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:40.418412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 183/364] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jun 7 08:21:40 2022] GET /identity/v3/role_assignments?group.id=0a07d620279f4fcab3723e3460d7d2bc&scope.project.id=d00b803196664866a4fd2eb970b1be0b&role.id=2bdabfafa05e443e842fe34d5f03e2c9 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:40.420422 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16263583-a65f-4bfa-b07f-e41679b673bf None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:40.457252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad9223-b104-40de-97d5-738229a1a491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:40.458035 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad9223-b104-40de-97d5-738229a1a491 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:40.458322 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad9223-b104-40de-97d5-738229a1a491 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:40.458604 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad9223-b104-40de-97d5-738229a1a491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:40.562262 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad9223-b104-40de-97d5-738229a1a491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:40.563850 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad9223-b104-40de-97d5-738229a1a491 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:40.569930 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16263583-a65f-4bfa-b07f-e41679b673bf None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:40.579258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:40.633356 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77154e-fd32-4d65-a64f-365de6e95d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:40.634046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77154e-fd32-4d65-a64f-365de6e95d99 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:40.634330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77154e-fd32-4d65-a64f-365de6e95d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:40.634607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77154e-fd32-4d65-a64f-365de6e95d99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:40.635793 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77154e-fd32-4d65-a64f-365de6e95d99 None admin] RBAC: Adding query filter params (role.id=6a4f5f6d39ed47f4a5036b99c7386949, user.id=d6b63f5565a84cbaaf4a3cbc925772a7, scope.system=all) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:40.636105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77154e-fd32-4d65-a64f-365de6e95d99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:40.639299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77154e-fd32-4d65-a64f-365de6e95d99 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:40.657108 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 184/365] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Jun 7 08:21:40 2022] GET /identity/v3/role_assignments?user.id=d6b63f5565a84cbaaf4a3cbc925772a7&scope.system=all&role.id=6a4f5f6d39ed47f4a5036b99c7386949 => generated 510 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:40.658477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53ca9-da9c-40a9-95ba-4ccd786504dd None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:40.658769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53ca9-da9c-40a9-95ba-4ccd786504dd None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:40.659051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53ca9-da9c-40a9-95ba-4ccd786504dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:40.680665 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5bb53ca9-da9c-40a9-95ba-4ccd786504dd None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:40.682140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5bb53ca9-da9c-40a9-95ba-4ccd786504dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:40.751224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 182/366] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:40.752058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad9223-b104-40de-97d5-738229a1a491 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:40.770868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:40.837237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 185/367] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:40.838254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb53ca9-da9c-40a9-95ba-4ccd786504dd None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:40.892639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad9223-b104-40de-97d5-738229a1a491 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:40.904835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:40.960118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca635c7-ec89-45bc-88f1-76f057767d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:40.960788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca635c7-ec89-45bc-88f1-76f057767d8e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:40.961076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca635c7-ec89-45bc-88f1-76f057767d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:40.961380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca635c7-ec89-45bc-88f1-76f057767d8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:41.058893 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb53ca9-da9c-40a9-95ba-4ccd786504dd None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:41.094084 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca635c7-ec89-45bc-88f1-76f057767d8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:41.095987 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca635c7-ec89-45bc-88f1-76f057767d8e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:41.109520 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaef582-cf55-4591-97fe-7ffff35c4912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:41.111017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaef582-cf55-4591-97fe-7ffff35c4912 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:41.111554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaef582-cf55-4591-97fe-7ffff35c4912 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:41.112001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaef582-cf55-4591-97fe-7ffff35c4912 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:41.114090 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaef582-cf55-4591-97fe-7ffff35c4912 None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:41.119625 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaef582-cf55-4591-97fe-7ffff35c4912 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:41.123491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 186/368] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:40 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:41.126835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaef582-cf55-4591-97fe-7ffff35c4912 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:41.201529 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:41.249233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 183/369] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:41.250832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca635c7-ec89-45bc-88f1-76f057767d8e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:41.265775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:41.313185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaef582-cf55-4591-97fe-7ffff35c4912 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:41.361144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af5b01-eb61-4972-ae0a-8584dd2e9c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:41.363326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af5b01-eb61-4972-ae0a-8584dd2e9c0b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:41.363762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af5b01-eb61-4972-ae0a-8584dd2e9c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:41.364178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af5b01-eb61-4972-ae0a-8584dd2e9c0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:41.379057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca635c7-ec89-45bc-88f1-76f057767d8e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:41.380200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af5b01-eb61-4972-ae0a-8584dd2e9c0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-00af5b01-eb61-4972-ae0a-8584dd2e9c0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:41.391856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:41.392980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:41.393889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:41.393889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:41.393889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:41.393889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:41.393889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:41.393889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:41.393889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:41.393889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:41.393889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:41.393889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:41.393889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:41.393889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:41.396650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 187/370] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:41.397775 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af5b01-eb61-4972-ae0a-8584dd2e9c0b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:41.425874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87611466-ee41-442d-bb7b-3357b0e42f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:41.426904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87611466-ee41-442d-bb7b-3357b0e42f15 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:41.427290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87611466-ee41-442d-bb7b-3357b0e42f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:41.427660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87611466-ee41-442d-bb7b-3357b0e42f15 None admin] PATH_INFO: `/v3/groups/0a07d620279f4fcab3723e3460d7d2bc` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:41.430666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87611466-ee41-442d-bb7b-3357b0e42f15 None admin] RBAC: Authorizing `identity:get_group(group_id=0a07d620279f4fcab3723e3460d7d2bc)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:41.431270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87611466-ee41-442d-bb7b-3357b0e42f15 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:41.433766 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 184/371] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:41 2022] GET /identity/v3/groups/0a07d620279f4fcab3723e3460d7d2bc => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:41.435301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87611466-ee41-442d-bb7b-3357b0e42f15 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:41.566294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:41.575572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af5b01-eb61-4972-ae0a-8584dd2e9c0b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:41.594190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:41.603412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87611466-ee41-442d-bb7b-3357b0e42f15 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:41.628596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e011101-1942-4fa0-b5bb-c6d323c4aab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:41.629957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e011101-1942-4fa0-b5bb-c6d323c4aab5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:41.630422 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e011101-1942-4fa0-b5bb-c6d323c4aab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:41.630883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e011101-1942-4fa0-b5bb-c6d323c4aab5 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:41.632359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e011101-1942-4fa0-b5bb-c6d323c4aab5 None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:41.634200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e011101-1942-4fa0-b5bb-c6d323c4aab5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:41.636214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 188/372] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:41 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:41.637337 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e011101-1942-4fa0-b5bb-c6d323c4aab5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:41.650341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14a7a8-af18-4d99-8b24-56fd2b4248d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:41.651142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14a7a8-af18-4d99-8b24-56fd2b4248d4 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:41.651581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14a7a8-af18-4d99-8b24-56fd2b4248d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:41.651927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14a7a8-af18-4d99-8b24-56fd2b4248d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:41.652751 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14a7a8-af18-4d99-8b24-56fd2b4248d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:41.653170 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14a7a8-af18-4d99-8b24-56fd2b4248d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:41.655219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14a7a8-af18-4d99-8b24-56fd2b4248d4 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:41.665179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 185/373] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:41.668523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14a7a8-af18-4d99-8b24-56fd2b4248d4 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:41.783668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e011101-1942-4fa0-b5bb-c6d323c4aab5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:41.797714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14a7a8-af18-4d99-8b24-56fd2b4248d4 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:41.844175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6b63e-a92d-42d0-ad95-49530d4a852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:41.844938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6b63e-a92d-42d0-ad95-49530d4a852f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:41.845390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6b63e-a92d-42d0-ad95-49530d4a852f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:41.845734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6b63e-a92d-42d0-ad95-49530d4a852f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:41.846682 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6b63e-a92d-42d0-ad95-49530d4a852f None admin] RBAC: Adding query filter params (group.id=0a07d620279f4fcab3723e3460d7d2bc, role.id=5addc8ffe20f47749792ac1394f6235c, scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:41.847164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6b63e-a92d-42d0-ad95-49530d4a852f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:41.848669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e8310-c7cc-44bc-8bdc-77007294cf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:41.848789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6b63e-a92d-42d0-ad95-49530d4a852f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:41.849846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e8310-c7cc-44bc-8bdc-77007294cf13 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:41.850300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e8310-c7cc-44bc-8bdc-77007294cf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:41.850851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e8310-c7cc-44bc-8bdc-77007294cf13 None admin] PATH_INFO: `/v3/groups/0a07d620279f4fcab3723e3460d7d2bc` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:41.852498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e8310-c7cc-44bc-8bdc-77007294cf13 None admin] RBAC: Authorizing `identity:get_group(group_id=0a07d620279f4fcab3723e3460d7d2bc)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:41.855235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e8310-c7cc-44bc-8bdc-77007294cf13 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:41.855450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 189/374] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jun 7 08:21:41 2022] GET /identity/v3/role_assignments?group.id=0a07d620279f4fcab3723e3460d7d2bc&scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b&role.id=5addc8ffe20f47749792ac1394f6235c => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:41.856291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6b63e-a92d-42d0-ad95-49530d4a852f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:41.858196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 186/375] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:41 2022] GET /identity/v3/groups/0a07d620279f4fcab3723e3460d7d2bc => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:41.865051 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e8310-c7cc-44bc-8bdc-77007294cf13 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:41.879374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:41.986934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:42.048739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e8310-c7cc-44bc-8bdc-77007294cf13 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:42.086348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6b63e-a92d-42d0-ad95-49530d4a852f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:42.105511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb5a97-9b74-4423-bea6-105bbcdd1d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:42.107199 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb5a97-9b74-4423-bea6-105bbcdd1d37 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:42.108528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb5a97-9b74-4423-bea6-105bbcdd1d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:42.108992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb5a97-9b74-4423-bea6-105bbcdd1d37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:42.110508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb5a97-9b74-4423-bea6-105bbcdd1d37 None admin] RBAC: Adding query filter params (group.id=0a07d620279f4fcab3723e3460d7d2bc, role.id=2bdabfafa05e443e842fe34d5f03e2c9, scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:42.111065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb5a97-9b74-4423-bea6-105bbcdd1d37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:42.113395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb5a97-9b74-4423-bea6-105bbcdd1d37 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:42.123388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 187/376] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jun 7 08:21:41 2022] GET /identity/v3/role_assignments?group.id=0a07d620279f4fcab3723e3460d7d2bc&scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b&role.id=2bdabfafa05e443e842fe34d5f03e2c9 => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:42.125041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cc712-7ee2-4875-9c45-e84aa124565f None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:42.125361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cc712-7ee2-4875-9c45-e84aa124565f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:42.125635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cc712-7ee2-4875-9c45-e84aa124565f None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:42.127214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 188/377] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:42.132634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b13e4-b69b-49ac-b32f-7ea418864bec None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:42.132914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b13e4-b69b-49ac-b32f-7ea418864bec None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:42.134136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b13e4-b69b-49ac-b32f-7ea418864bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:42.148325 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e9ad6-6e29-4841-90f5-de2d18781c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:42.149019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e9ad6-6e29-4841-90f5-de2d18781c91 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:42.149335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e9ad6-6e29-4841-90f5-de2d18781c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:42.149614 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e9ad6-6e29-4841-90f5-de2d18781c91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:42.154498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-ef1b13e4-b69b-49ac-b32f-7ea418864bec None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:42.156542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ef1b13e4-b69b-49ac-b32f-7ea418864bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:42.269142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e9ad6-6e29-4841-90f5-de2d18781c91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:42.269142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e9ad6-6e29-4841-90f5-de2d18781c91 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:42.273708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 189/378] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:42.338077 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:42.387155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b13e4-b69b-49ac-b32f-7ea418864bec None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:42.389989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:42.418874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 190/379] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:42.425943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e9ad6-6e29-4841-90f5-de2d18781c91 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:42.513792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b13e4-b69b-49ac-b32f-7ea418864bec None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:42.566648 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5bb19-d1c5-4770-8e31-e34a90bb9296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:42.567791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5bb19-d1c5-4770-8e31-e34a90bb9296 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:42.568257 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5bb19-d1c5-4770-8e31-e34a90bb9296 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:42.568737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5bb19-d1c5-4770-8e31-e34a90bb9296 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:42.591216 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5bb19-d1c5-4770-8e31-e34a90bb9296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-07d5bb19-d1c5-4770-8e31-e34a90bb9296 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:42.604197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:42.605281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:42.606155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:42.606155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:42.606155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:42.606155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:42.606155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:42.606155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:42.606155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:42.606155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:42.606155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:42.606155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:42.606155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:42.606155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:42.606155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 190/380] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:21:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:42.613570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5bb19-d1c5-4770-8e31-e34a90bb9296 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:42.627523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e9ad6-6e29-4841-90f5-de2d18781c91 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:42.696757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:42.748248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5bb19-d1c5-4770-8e31-e34a90bb9296 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:42.751278 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9535b2-dc22-4bff-a7a0-b81ae8a3dab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:42.752500 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9535b2-dc22-4bff-a7a0-b81ae8a3dab4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:42.752997 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9535b2-dc22-4bff-a7a0-b81ae8a3dab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:42.753507 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9535b2-dc22-4bff-a7a0-b81ae8a3dab4 None admin] PATH_INFO: `/v3/projects/ddc110e64f52436a913830733ab91055` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:42.759339 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9535b2-dc22-4bff-a7a0-b81ae8a3dab4 None admin] RBAC: Authorizing `identity:get_project(project_id=ddc110e64f52436a913830733ab91055)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:42.761273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9535b2-dc22-4bff-a7a0-b81ae8a3dab4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:42.769273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 191/381] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:42 2022] GET /identity/v3/projects/ddc110e64f52436a913830733ab91055 => generated 344 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:42.774945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9535b2-dc22-4bff-a7a0-b81ae8a3dab4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:42.801490 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:42.845376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88fe17-e2c3-41bd-a0b0-9de28f639091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:42.846069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88fe17-e2c3-41bd-a0b0-9de28f639091 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:42.846358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88fe17-e2c3-41bd-a0b0-9de28f639091 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:42.846636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88fe17-e2c3-41bd-a0b0-9de28f639091 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:42.847406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88fe17-e2c3-41bd-a0b0-9de28f639091 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:42.847723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88fe17-e2c3-41bd-a0b0-9de28f639091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:42.849171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88fe17-e2c3-41bd-a0b0-9de28f639091 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:42.864686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 191/382] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:42.871978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88fe17-e2c3-41bd-a0b0-9de28f639091 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:42.964129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9535b2-dc22-4bff-a7a0-b81ae8a3dab4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:43.019439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17a5a5-3153-4ffc-bbe9-70d1aabe3467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:43.021531 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17a5a5-3153-4ffc-bbe9-70d1aabe3467 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:43.022517 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17a5a5-3153-4ffc-bbe9-70d1aabe3467 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:43.023132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17a5a5-3153-4ffc-bbe9-70d1aabe3467 None admin] PATH_INFO: `/v3/groups/7f125bf1255248b7a99fca7d6ba93a48` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:43.035746 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17a5a5-3153-4ffc-bbe9-70d1aabe3467 None admin] RBAC: Authorizing `identity:get_group(group_id=7f125bf1255248b7a99fca7d6ba93a48)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:43.039872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17a5a5-3153-4ffc-bbe9-70d1aabe3467 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:43.043796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 192/383] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:42 2022] GET /identity/v3/groups/7f125bf1255248b7a99fca7d6ba93a48 => generated 229 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:43.055536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17a5a5-3153-4ffc-bbe9-70d1aabe3467 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:43.115275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88fe17-e2c3-41bd-a0b0-9de28f639091 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:43.167442 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:43.201731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bc124-56c0-4adb-be8d-1453cbd854f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:43.202856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bc124-56c0-4adb-be8d-1453cbd854f5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:43.203461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bc124-56c0-4adb-be8d-1453cbd854f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:43.203835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bc124-56c0-4adb-be8d-1453cbd854f5 None admin] PATH_INFO: `/v3/groups/0a07d620279f4fcab3723e3460d7d2bc` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:43.210101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bc124-56c0-4adb-be8d-1453cbd854f5 None admin] RBAC: Authorizing `identity:get_group(group_id=0a07d620279f4fcab3723e3460d7d2bc)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:43.211876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bc124-56c0-4adb-be8d-1453cbd854f5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:43.214273 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 192/384] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:42 2022] GET /identity/v3/groups/0a07d620279f4fcab3723e3460d7d2bc => generated 226 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:43.223928 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bc124-56c0-4adb-be8d-1453cbd854f5 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:43.242532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:43.291866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17a5a5-3153-4ffc-bbe9-70d1aabe3467 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:43.335087 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0186d7-3c69-4f08-804c-7122cd6c8a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:43.336234 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0186d7-3c69-4f08-804c-7122cd6c8a72 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:43.336723 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0186d7-3c69-4f08-804c-7122cd6c8a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:43.337263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0186d7-3c69-4f08-804c-7122cd6c8a72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:43.338346 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0186d7-3c69-4f08-804c-7122cd6c8a72 None admin] RBAC: Adding query filter params (group.id=7f125bf1255248b7a99fca7d6ba93a48, role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=ddc110e64f52436a913830733ab91055) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:43.338857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0186d7-3c69-4f08-804c-7122cd6c8a72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:43.340499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0186d7-3c69-4f08-804c-7122cd6c8a72 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:43.349972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 193/385] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jun 7 08:21:43 2022] GET /identity/v3/role_assignments?group.id=7f125bf1255248b7a99fca7d6ba93a48&scope.project.id=ddc110e64f52436a913830733ab91055&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 260 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:43.442326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bc124-56c0-4adb-be8d-1453cbd854f5 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:43.505889 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697dab9-f69f-44a8-8576-edce4328df38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:43.506703 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697dab9-f69f-44a8-8576-edce4328df38 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:43.506990 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697dab9-f69f-44a8-8576-edce4328df38 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:43.507270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697dab9-f69f-44a8-8576-edce4328df38 None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:43.510320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697dab9-f69f-44a8-8576-edce4328df38 None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:43.512714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697dab9-f69f-44a8-8576-edce4328df38 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:43.516480 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 193/386] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:43 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:43.522570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0186d7-3c69-4f08-804c-7122cd6c8a72 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:43.678088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0186d7-3c69-4f08-804c-7122cd6c8a72 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:43.704761 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0c651-e854-496c-81d1-47c20218db71 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:43.705114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0c651-e854-496c-81d1-47c20218db71 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:43.705712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0c651-e854-496c-81d1-47c20218db71 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:43.713939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 194/387] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:43 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:43.722655 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775b5d3-5511-4a07-9e86-4281bf86d5bb None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:43.723317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775b5d3-5511-4a07-9e86-4281bf86d5bb None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:43.723673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579d3fe-7595-4ea4-a3f6-a44e5f23a03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:43.724049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775b5d3-5511-4a07-9e86-4281bf86d5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:43.724566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579d3fe-7595-4ea4-a3f6-a44e5f23a03f None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:43.724897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579d3fe-7595-4ea4-a3f6-a44e5f23a03f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:43.725210 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579d3fe-7595-4ea4-a3f6-a44e5f23a03f None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b/groups/0a07d620279f4fcab3723e3460d7d2bc/roles/5addc8ffe20f47749792ac1394f6235c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:43.727596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579d3fe-7595-4ea4-a3f6-a44e5f23a03f None admin] RBAC: Authorizing `identity:create_grant(project_id=d00b803196664866a4fd2eb970b1be0b, group_id=0a07d620279f4fcab3723e3460d7d2bc, role_id=5addc8ffe20f47749792ac1394f6235c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:43.733190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579d3fe-7595-4ea4-a3f6-a44e5f23a03f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:43.740720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:43.749454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 194/388] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Jun 7 08:21:43 2022] PUT /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b/groups/0a07d620279f4fcab3723e3460d7d2bc/roles/5addc8ffe20f47749792ac1394f6235c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:43.765640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-f775b5d3-5511-4a07-9e86-4281bf86d5bb None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:43.767194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f775b5d3-5511-4a07-9e86-4281bf86d5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:43.888234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:43.905982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 195/389] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:43.953744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579d3fe-7595-4ea4-a3f6-a44e5f23a03f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:44.109937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579d3fe-7595-4ea4-a3f6-a44e5f23a03f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:44.152735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716f0a4-fa1d-4656-b177-e300e71e592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:44.153810 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716f0a4-fa1d-4656-b177-e300e71e592b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:44.154302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716f0a4-fa1d-4656-b177-e300e71e592b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:44.154837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716f0a4-fa1d-4656-b177-e300e71e592b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:44.167850 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716f0a4-fa1d-4656-b177-e300e71e592b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-d716f0a4-fa1d-4656-b177-e300e71e592b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:44.176745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:44.177663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:44.178473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:44.178473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:44.178473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:44.178473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:44.178473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:44.178473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:44.178473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:44.178473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:44.178473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:44.178473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:44.178473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:44.178473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:44.178473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 195/390] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:44.184039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775b5d3-5511-4a07-9e86-4281bf86d5bb None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:44.402923 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775b5d3-5511-4a07-9e86-4281bf86d5bb None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:44.500407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187cab1-0c8d-4392-87fd-24878107d372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:44.501192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187cab1-0c8d-4392-87fd-24878107d372 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:44.501407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187cab1-0c8d-4392-87fd-24878107d372 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:44.501689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187cab1-0c8d-4392-87fd-24878107d372 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:44.502455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187cab1-0c8d-4392-87fd-24878107d372 None admin] RBAC: Adding query filter params (name=member) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:44.502793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187cab1-0c8d-4392-87fd-24878107d372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:44.504246 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187cab1-0c8d-4392-87fd-24878107d372 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:44.517962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 196/391] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:44.530101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716f0a4-fa1d-4656-b177-e300e71e592b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:44.664007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:44.787748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716f0a4-fa1d-4656-b177-e300e71e592b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:44.865796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c51f04-c5ad-481d-b100-66a3235b5085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:44.866698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c51f04-c5ad-481d-b100-66a3235b5085 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:44.866983 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c51f04-c5ad-481d-b100-66a3235b5085 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:44.867263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c51f04-c5ad-481d-b100-66a3235b5085 None admin] PATH_INFO: `/v3/groups/0a07d620279f4fcab3723e3460d7d2bc` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:44.869211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c51f04-c5ad-481d-b100-66a3235b5085 None admin] RBAC: Authorizing `identity:get_group(group_id=0a07d620279f4fcab3723e3460d7d2bc)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:44.870718 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c51f04-c5ad-481d-b100-66a3235b5085 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:44.872895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 196/392] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:44 2022] GET /identity/v3/groups/0a07d620279f4fcab3723e3460d7d2bc => generated 226 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:44.879858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187cab1-0c8d-4392-87fd-24878107d372 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:45.038908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:45.148085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187cab1-0c8d-4392-87fd-24878107d372 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:45.231053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd8106-84fe-45f7-9afe-5f31d4b6ae5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:45.232563 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd8106-84fe-45f7-9afe-5f31d4b6ae5f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:45.232978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd8106-84fe-45f7-9afe-5f31d4b6ae5f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:45.233241 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd8106-84fe-45f7-9afe-5f31d4b6ae5f None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:45.238015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd8106-84fe-45f7-9afe-5f31d4b6ae5f None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:45.239513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd8106-84fe-45f7-9afe-5f31d4b6ae5f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:45.241685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 197/393] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:44 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:45.253383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c51f04-c5ad-481d-b100-66a3235b5085 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:45.288443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7b115-874d-4500-b483-3dbcb6685e67 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:45.289173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7b115-874d-4500-b483-3dbcb6685e67 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:45.289670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7b115-874d-4500-b483-3dbcb6685e67 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:45.292190 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 198/394] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:45.298270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d13af-e84c-4306-a42c-8ca7edf977af None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:45.298856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d13af-e84c-4306-a42c-8ca7edf977af None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:45.299362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d13af-e84c-4306-a42c-8ca7edf977af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:45.326069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1b7d13af-e84c-4306-a42c-8ca7edf977af None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:45.330058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1b7d13af-e84c-4306-a42c-8ca7edf977af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:45.470909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 199/395] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:45.480182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c51f04-c5ad-481d-b100-66a3235b5085 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:45.519451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d13af-e84c-4306-a42c-8ca7edf977af None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:45.525549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:45.545005 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec632dde-a83d-42cb-9153-8f05aec1ea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:45.545901 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec632dde-a83d-42cb-9153-8f05aec1ea9c None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:45.546184 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec632dde-a83d-42cb-9153-8f05aec1ea9c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:45.546470 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec632dde-a83d-42cb-9153-8f05aec1ea9c None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b/groups/0a07d620279f4fcab3723e3460d7d2bc/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:45.551868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec632dde-a83d-42cb-9153-8f05aec1ea9c None admin] RBAC: Authorizing `identity:create_grant(project_id=d00b803196664866a4fd2eb970b1be0b, group_id=0a07d620279f4fcab3723e3460d7d2bc, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:45.556090 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec632dde-a83d-42cb-9153-8f05aec1ea9c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:45.577755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 197/396] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Jun 7 08:21:45 2022] PUT /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b/groups/0a07d620279f4fcab3723e3460d7d2bc/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:45.580285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf468c-95d2-43ae-93b4-7b59f73502fe None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:45.580774 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf468c-95d2-43ae-93b4-7b59f73502fe None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:45.581360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf468c-95d2-43ae-93b4-7b59f73502fe None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:45.583332 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 198/397] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:45.593988 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf6261-2668-439e-80e2-1db4da60f554 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:45.594514 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf6261-2668-439e-80e2-1db4da60f554 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:45.594966 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf6261-2668-439e-80e2-1db4da60f554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:45.633920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7aaf6261-2668-439e-80e2-1db4da60f554 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:45.637112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7aaf6261-2668-439e-80e2-1db4da60f554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:45.638753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:45.725727 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d13af-e84c-4306-a42c-8ca7edf977af None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:45.767756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b226715-be59-42e3-95c0-62d71bd59fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:45.769439 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b226715-be59-42e3-95c0-62d71bd59fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:45.769899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b226715-be59-42e3-95c0-62d71bd59fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:45.770319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b226715-be59-42e3-95c0-62d71bd59fd5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:45.784049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b226715-be59-42e3-95c0-62d71bd59fd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-2b226715-be59-42e3-95c0-62d71bd59fd5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:45.794705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:45.795844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:45.796775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:45.796775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:45.796775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:45.796775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:45.796775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:45.796775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:45.796775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:45.796775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:45.796775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:45.796775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:45.796775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:45.796775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:45.800009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 200/398] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:21:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:45.806146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b226715-be59-42e3-95c0-62d71bd59fd5 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:45.834520 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 199/399] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 07 08:21:45.858660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf6261-2668-439e-80e2-1db4da60f554 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:45.946637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b226715-be59-42e3-95c0-62d71bd59fd5 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:45.960886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:45.993792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77711b02-2b0b-4872-8b72-ee7653505af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:45.994482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77711b02-2b0b-4872-8b72-ee7653505af0 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:45.994764 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77711b02-2b0b-4872-8b72-ee7653505af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:45.995039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77711b02-2b0b-4872-8b72-ee7653505af0 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:45.995868 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77711b02-2b0b-4872-8b72-ee7653505af0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:45.996226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77711b02-2b0b-4872-8b72-ee7653505af0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:45.998323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77711b02-2b0b-4872-8b72-ee7653505af0 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:46.009816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 201/400] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:46.015787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf6261-2668-439e-80e2-1db4da60f554 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:46.017375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77711b02-2b0b-4872-8b72-ee7653505af0 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:46.041785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:46.070883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae4c3e-691d-47cf-aa01-f3aa32099564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:46.072194 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae4c3e-691d-47cf-aa01-f3aa32099564 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.072727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae4c3e-691d-47cf-aa01-f3aa32099564 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.073407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae4c3e-691d-47cf-aa01-f3aa32099564 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.086970 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae4c3e-691d-47cf-aa01-f3aa32099564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-dfae4c3e-691d-47cf-aa01-f3aa32099564 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:46.096386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:46.097373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:46.098146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:46.098146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:46.098146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:46.098146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:46.098146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:46.098146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:46.098146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:46.098146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:46.098146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:46.098146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:46.098146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:46.098146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:46.101129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 200/401] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:46.110610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae4c3e-691d-47cf-aa01-f3aa32099564 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:46.194104 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77711b02-2b0b-4872-8b72-ee7653505af0 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:46.269071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e898c27-87e7-4ea9-8d09-973ae39adcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:46.269953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e898c27-87e7-4ea9-8d09-973ae39adcac None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.270236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e898c27-87e7-4ea9-8d09-973ae39adcac None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.270508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e898c27-87e7-4ea9-8d09-973ae39adcac None admin] PATH_INFO: `/v3/groups/0a07d620279f4fcab3723e3460d7d2bc` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.272062 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e898c27-87e7-4ea9-8d09-973ae39adcac None admin] RBAC: Authorizing `identity:get_group(group_id=0a07d620279f4fcab3723e3460d7d2bc)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:46.274292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e898c27-87e7-4ea9-8d09-973ae39adcac None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:46.276018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 202/402] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:46 2022] GET /identity/v3/groups/0a07d620279f4fcab3723e3460d7d2bc => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:46.277268 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0eead-a16c-46c7-b7ff-960ddeef4c6e None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.277551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0eead-a16c-46c7-b7ff-960ddeef4c6e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.277822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0eead-a16c-46c7-b7ff-960ddeef4c6e None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.279629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 203/403] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:46.284481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0eead-a16c-46c7-b7ff-960ddeef4c6e None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:46.290748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae4c3e-691d-47cf-aa01-f3aa32099564 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:46.330307 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b31656-b3b6-4dee-92cb-fbd280a3a958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:46.331030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b31656-b3b6-4dee-92cb-fbd280a3a958 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.331377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b31656-b3b6-4dee-92cb-fbd280a3a958 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.331722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b31656-b3b6-4dee-92cb-fbd280a3a958 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.332535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b31656-b3b6-4dee-92cb-fbd280a3a958 None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:46.332906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b31656-b3b6-4dee-92cb-fbd280a3a958 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:46.334393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b31656-b3b6-4dee-92cb-fbd280a3a958 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:46.339006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:46.351667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 201/404] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:46.353251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da054d14-112b-41b0-b426-4a4cb8ece4dc None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.353727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da054d14-112b-41b0-b426-4a4cb8ece4dc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.354096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da054d14-112b-41b0-b426-4a4cb8ece4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.373792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-da054d14-112b-41b0-b426-4a4cb8ece4dc None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:46.375426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-da054d14-112b-41b0-b426-4a4cb8ece4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:46.410180 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:46.416384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0eead-a16c-46c7-b7ff-960ddeef4c6e None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:46.461297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3928570-46a9-43a3-ba85-89eb015cb4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:46.461297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3928570-46a9-43a3-ba85-89eb015cb4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.461297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3928570-46a9-43a3-ba85-89eb015cb4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.461297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3928570-46a9-43a3-ba85-89eb015cb4d3 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.461693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3928570-46a9-43a3-ba85-89eb015cb4d3 None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:46.461939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3928570-46a9-43a3-ba85-89eb015cb4d3 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:46.462133 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 202/405] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:46.463253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da054d14-112b-41b0-b426-4a4cb8ece4dc None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:46.468763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 204/406] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:46 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:46.470621 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3928570-46a9-43a3-ba85-89eb015cb4d3 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:46.590645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3928570-46a9-43a3-ba85-89eb015cb4d3 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:46.601815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:46.612932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da054d14-112b-41b0-b426-4a4cb8ece4dc None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:46.634023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88749191-1cf8-433b-90e3-85a8968da5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:46.635190 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88749191-1cf8-433b-90e3-85a8968da5f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.635301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88749191-1cf8-433b-90e3-85a8968da5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.635549 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88749191-1cf8-433b-90e3-85a8968da5f0 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b/groups/0a07d620279f4fcab3723e3460d7d2bc/roles/5addc8ffe20f47749792ac1394f6235c` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.638612 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88749191-1cf8-433b-90e3-85a8968da5f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=a49ddfabaeee47e0ad65b7bf6470e03b, group_id=0a07d620279f4fcab3723e3460d7d2bc, role_id=5addc8ffe20f47749792ac1394f6235c)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:46.640362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88749191-1cf8-433b-90e3-85a8968da5f0 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:46.661224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 205/407] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Jun 7 08:21:46 2022] PUT /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b/groups/0a07d620279f4fcab3723e3460d7d2bc/roles/5addc8ffe20f47749792ac1394f6235c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:46.662482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88749191-1cf8-433b-90e3-85a8968da5f0 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:46.669138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c836310-53d1-4649-8e6c-4dfd8ea46745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:46.670911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c836310-53d1-4649-8e6c-4dfd8ea46745 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.671507 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c836310-53d1-4649-8e6c-4dfd8ea46745 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.672003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c836310-53d1-4649-8e6c-4dfd8ea46745 None admin] PATH_INFO: `/v3/groups/0a07d620279f4fcab3723e3460d7d2bc` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.674819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c836310-53d1-4649-8e6c-4dfd8ea46745 None admin] RBAC: Authorizing `identity:get_group(group_id=0a07d620279f4fcab3723e3460d7d2bc)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:46.679619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c836310-53d1-4649-8e6c-4dfd8ea46745 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:46.685395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 203/408] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:46 2022] GET /identity/v3/groups/0a07d620279f4fcab3723e3460d7d2bc => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:46.691806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c836310-53d1-4649-8e6c-4dfd8ea46745 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:46.771397 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:46.888269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88749191-1cf8-433b-90e3-85a8968da5f0 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:46.903981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c836310-53d1-4649-8e6c-4dfd8ea46745 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:46.963382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81456c-6da3-4c8e-a7d5-00865dc8ecc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:46.964191 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81456c-6da3-4c8e-a7d5-00865dc8ecc4 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.964474 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81456c-6da3-4c8e-a7d5-00865dc8ecc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.964741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81456c-6da3-4c8e-a7d5-00865dc8ecc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.969710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f22bb-87cb-47af-b439-e99e317c2af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:46.970939 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f22bb-87cb-47af-b439-e99e317c2af1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.971446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f22bb-87cb-47af-b439-e99e317c2af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.971922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f22bb-87cb-47af-b439-e99e317c2af1 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.973472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f22bb-87cb-47af-b439-e99e317c2af1 None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:46.976092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f22bb-87cb-47af-b439-e99e317c2af1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:46.978400 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81456c-6da3-4c8e-a7d5-00865dc8ecc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:46.978870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 204/409] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:46 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-3b81456c-6da3-4c8e-a7d5-00865dc8ecc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:46.989995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:46.990969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:46.991756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:46.991756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:46.991756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:46.991756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:46.991756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:46.991756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:46.991756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:46.991756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:46.991756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:46.991756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:46.991756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:46.991756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:46.992349 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f22bb-87cb-47af-b439-e99e317c2af1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:46.994706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 206/410] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:47.003894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81456c-6da3-4c8e-a7d5-00865dc8ecc4 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:47.042792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:47.149361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81456c-6da3-4c8e-a7d5-00865dc8ecc4 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:47.172618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f22bb-87cb-47af-b439-e99e317c2af1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:47.177003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:47.230757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066b806-8016-44dd-906a-01fd1145990f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:47.231987 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066b806-8016-44dd-906a-01fd1145990f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:47.232280 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066b806-8016-44dd-906a-01fd1145990f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:47.232644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066b806-8016-44dd-906a-01fd1145990f None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:47.233538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066b806-8016-44dd-906a-01fd1145990f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:47.233849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066b806-8016-44dd-906a-01fd1145990f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:47.235279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066b806-8016-44dd-906a-01fd1145990f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:47.252848 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0d601-48e9-4549-85ac-d6b5f417cf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:47.253792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0d601-48e9-4549-85ac-d6b5f417cf3b None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:47.254077 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0d601-48e9-4549-85ac-d6b5f417cf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:47.254354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0d601-48e9-4549-85ac-d6b5f417cf3b None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b/groups/0a07d620279f4fcab3723e3460d7d2bc/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:47.258050 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 207/411] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:47.258763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0d601-48e9-4549-85ac-d6b5f417cf3b None admin] RBAC: Authorizing `identity:create_grant(project_id=a49ddfabaeee47e0ad65b7bf6470e03b, group_id=0a07d620279f4fcab3723e3460d7d2bc, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:47.260393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0d601-48e9-4549-85ac-d6b5f417cf3b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:47.268656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066b806-8016-44dd-906a-01fd1145990f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:47.287781 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 205/412] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Jun 7 08:21:46 2022] PUT /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b/groups/0a07d620279f4fcab3723e3460d7d2bc/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:21:47.415511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:47.427809 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066b806-8016-44dd-906a-01fd1145990f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:47.466730 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3903cf3-fea2-4d13-805c-2ec9bb912a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:47.468285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3903cf3-fea2-4d13-805c-2ec9bb912a2f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:47.468772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3903cf3-fea2-4d13-805c-2ec9bb912a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:47.469310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3903cf3-fea2-4d13-805c-2ec9bb912a2f None admin] PATH_INFO: `/v3/groups/7f125bf1255248b7a99fca7d6ba93a48` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:47.470890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3903cf3-fea2-4d13-805c-2ec9bb912a2f None admin] RBAC: Authorizing `identity:get_group(group_id=7f125bf1255248b7a99fca7d6ba93a48)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:47.472805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3903cf3-fea2-4d13-805c-2ec9bb912a2f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:47.475196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 208/413] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:47 2022] GET /identity/v3/groups/7f125bf1255248b7a99fca7d6ba93a48 => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:47.481028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0d601-48e9-4549-85ac-d6b5f417cf3b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:47.516759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d7fea-7b98-479e-aeaf-9d62a4bff443 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:47.517419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d7fea-7b98-479e-aeaf-9d62a4bff443 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:47.517924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d7fea-7b98-479e-aeaf-9d62a4bff443 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:47.519880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 209/414] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:47.525245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bb880-7834-4d90-b60f-8695e4e65941 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:47.525626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bb880-7834-4d90-b60f-8695e4e65941 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:47.525815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bb880-7834-4d90-b60f-8695e4e65941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:47.544569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-541bb880-7834-4d90-b60f-8695e4e65941 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:47.545978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-541bb880-7834-4d90-b60f-8695e4e65941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:47.670978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 210/415] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:47.696701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bb880-7834-4d90-b60f-8695e4e65941 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:47.697507 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0d601-48e9-4549-85ac-d6b5f417cf3b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:47.741695 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7517198-d75d-4515-9c83-add42980f63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:47.742572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7517198-d75d-4515-9c83-add42980f63c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:47.742857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7517198-d75d-4515-9c83-add42980f63c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:47.743131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7517198-d75d-4515-9c83-add42980f63c None admin] PATH_INFO: `/v3/projects/ddc110e64f52436a913830733ab91055` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:47.744414 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7517198-d75d-4515-9c83-add42980f63c None admin] RBAC: Authorizing `identity:get_project(project_id=ddc110e64f52436a913830733ab91055)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:47.746186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7517198-d75d-4515-9c83-add42980f63c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:47.749570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 206/416] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:47 2022] GET /identity/v3/projects/ddc110e64f52436a913830733ab91055 => generated 344 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:47.755978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7517198-d75d-4515-9c83-add42980f63c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:47.831051 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:47.833631 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:47.861501 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bb880-7834-4d90-b60f-8695e4e65941 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:47.923981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7246b03-6b25-4381-a712-2b5753cc3cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:47.925167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7246b03-6b25-4381-a712-2b5753cc3cbe None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:47.925606 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7246b03-6b25-4381-a712-2b5753cc3cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:47.926053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7246b03-6b25-4381-a712-2b5753cc3cbe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:47.948222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7246b03-6b25-4381-a712-2b5753cc3cbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-e7246b03-6b25-4381-a712-2b5753cc3cbe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:47.962845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:47.963812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:47.964679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:47.964679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:47.964679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:47.964679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:47.964679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:47.964679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:47.964679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:47.964679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:47.964679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:47.964679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:47.964679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:47.964679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:47.976376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 211/417] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:21:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:47.977859 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7246b03-6b25-4381-a712-2b5753cc3cbe None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:48.003937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7517198-d75d-4515-9c83-add42980f63c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:48.082270 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7acc5-8464-495f-9b18-c746c076fabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:48.083182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7acc5-8464-495f-9b18-c746c076fabf None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:48.083534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7acc5-8464-495f-9b18-c746c076fabf None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:48.083811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7acc5-8464-495f-9b18-c746c076fabf None admin] PATH_INFO: `/v3/projects/ddc110e64f52436a913830733ab91055/groups/7f125bf1255248b7a99fca7d6ba93a48/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:48.086955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7acc5-8464-495f-9b18-c746c076fabf None admin] RBAC: Authorizing `identity:create_grant(project_id=ddc110e64f52436a913830733ab91055, group_id=7f125bf1255248b7a99fca7d6ba93a48, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:48.088471 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7acc5-8464-495f-9b18-c746c076fabf None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:48.113745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 207/418] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Jun 7 08:21:47 2022] PUT /identity/v3/projects/ddc110e64f52436a913830733ab91055/groups/7f125bf1255248b7a99fca7d6ba93a48/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:48.178135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7246b03-6b25-4381-a712-2b5753cc3cbe None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:48.221954 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9e8f6-afea-491d-ac93-f8e2e7d08972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:48.223042 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9e8f6-afea-491d-ac93-f8e2e7d08972 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:48.223531 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9e8f6-afea-491d-ac93-f8e2e7d08972 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:48.223998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9e8f6-afea-491d-ac93-f8e2e7d08972 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:48.224974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9e8f6-afea-491d-ac93-f8e2e7d08972 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:48.225529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9e8f6-afea-491d-ac93-f8e2e7d08972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:48.227188 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9e8f6-afea-491d-ac93-f8e2e7d08972 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:48.238553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 212/419] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:48.244245 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7acc5-8464-495f-9b18-c746c076fabf None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:48.393353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7077a65-a0dd-4769-a21a-bea5d3e2868c None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:48.393694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7077a65-a0dd-4769-a21a-bea5d3e2868c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:48.393924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7077a65-a0dd-4769-a21a-bea5d3e2868c None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:48.397005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 213/420] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:48.401731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd8fe5-2912-49e1-92b8-b919de46adc3 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:48.402060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd8fe5-2912-49e1-92b8-b919de46adc3 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:48.402305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd8fe5-2912-49e1-92b8-b919de46adc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:48.413912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:48.423932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7acc5-8464-495f-9b18-c746c076fabf None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:48.447834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d4bd8fe5-2912-49e1-92b8-b919de46adc3 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:48.449578 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d4bd8fe5-2912-49e1-92b8-b919de46adc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:48.454679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:48.476179 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fafd51-9376-4213-921b-b5cb3606dcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:48.477050 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fafd51-9376-4213-921b-b5cb3606dcde None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:48.477483 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fafd51-9376-4213-921b-b5cb3606dcde None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:48.477755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fafd51-9376-4213-921b-b5cb3606dcde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:48.553884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 214/421] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:48.583329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd8fe5-2912-49e1-92b8-b919de46adc3 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:48.622927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fafd51-9376-4213-921b-b5cb3606dcde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:48.624744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fafd51-9376-4213-921b-b5cb3606dcde None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:48.768337 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:48.769400 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 208/422] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:48.774336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fafd51-9376-4213-921b-b5cb3606dcde None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:48.867514 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd8fe5-2912-49e1-92b8-b919de46adc3 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:48.899594 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fafd51-9376-4213-921b-b5cb3606dcde None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:48.952138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0308ab4-24d4-4db3-81a7-e8afb33837fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:48.953847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0308ab4-24d4-4db3-81a7-e8afb33837fb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:48.954591 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0308ab4-24d4-4db3-81a7-e8afb33837fb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:48.955097 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0308ab4-24d4-4db3-81a7-e8afb33837fb None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:48.958613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0308ab4-24d4-4db3-81a7-e8afb33837fb None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:48.960482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0308ab4-24d4-4db3-81a7-e8afb33837fb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:48.962995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 209/423] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:48 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:48.969674 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0308ab4-24d4-4db3-81a7-e8afb33837fb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:48.998501 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1e60d-c654-4727-a4f1-4e470f68c941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:48.999936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1e60d-c654-4727-a4f1-4e470f68c941 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:48.999936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1e60d-c654-4727-a4f1-4e470f68c941 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:48.999936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1e60d-c654-4727-a4f1-4e470f68c941 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:49.034849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:49.049288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1e60d-c654-4727-a4f1-4e470f68c941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:49.056600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-efc1e60d-c654-4727-a4f1-4e470f68c941 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:49.062883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:49.063834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:49.064640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:49.064640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:49.064640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:49.064640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:49.064640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:49.064640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:49.064640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:49.064640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:49.064640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:49.064640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:49.064640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:49.064640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:49.073560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 215/424] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 491 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:49.082207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1e60d-c654-4727-a4f1-4e470f68c941 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:49.174881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0308ab4-24d4-4db3-81a7-e8afb33837fb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:49.271321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9216121-84a9-43ae-9fca-0c8f1b5a8c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:49.272398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9216121-84a9-43ae-9fca-0c8f1b5a8c87 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:49.272712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9216121-84a9-43ae-9fca-0c8f1b5a8c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:49.273049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9216121-84a9-43ae-9fca-0c8f1b5a8c87 None admin] PATH_INFO: `/v3/groups/0a07d620279f4fcab3723e3460d7d2bc` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:49.275148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9216121-84a9-43ae-9fca-0c8f1b5a8c87 None admin] RBAC: Authorizing `identity:get_group(group_id=0a07d620279f4fcab3723e3460d7d2bc)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:49.276717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9216121-84a9-43ae-9fca-0c8f1b5a8c87 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:49.279609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 210/425] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:48 2022] GET /identity/v3/groups/0a07d620279f4fcab3723e3460d7d2bc => generated 226 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:49.287176 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9216121-84a9-43ae-9fca-0c8f1b5a8c87 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:49.363282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1e60d-c654-4727-a4f1-4e470f68c941 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:49.456659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9216121-84a9-43ae-9fca-0c8f1b5a8c87 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:49.461843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b0da5-de00-4331-9a7f-5ba2ed67e4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:49.462535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b0da5-de00-4331-9a7f-5ba2ed67e4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:49.462815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b0da5-de00-4331-9a7f-5ba2ed67e4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:49.463095 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b0da5-de00-4331-9a7f-5ba2ed67e4e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:49.463857 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b0da5-de00-4331-9a7f-5ba2ed67e4e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:49.464172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b0da5-de00-4331-9a7f-5ba2ed67e4e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:49.465598 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b0da5-de00-4331-9a7f-5ba2ed67e4e8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:49.485151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 216/426] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:49.486376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:49.494524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b0da5-de00-4331-9a7f-5ba2ed67e4e8 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:49.502210 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42046fa-cf0d-4bbe-a9f3-e6ea5b856600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:49.504709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42046fa-cf0d-4bbe-a9f3-e6ea5b856600 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:49.506107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42046fa-cf0d-4bbe-a9f3-e6ea5b856600 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:49.509540 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42046fa-cf0d-4bbe-a9f3-e6ea5b856600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:49.511474 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42046fa-cf0d-4bbe-a9f3-e6ea5b856600 None admin] RBAC: Adding query filter params (group.id=0a07d620279f4fcab3723e3460d7d2bc, role.id=5addc8ffe20f47749792ac1394f6235c, scope.project.id=d00b803196664866a4fd2eb970b1be0b) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:49.514795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42046fa-cf0d-4bbe-a9f3-e6ea5b856600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:49.517682 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42046fa-cf0d-4bbe-a9f3-e6ea5b856600 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:49.532493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 211/427] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jun 7 08:21:49 2022] GET /identity/v3/role_assignments?group.id=0a07d620279f4fcab3723e3460d7d2bc&scope.project.id=d00b803196664866a4fd2eb970b1be0b&role.id=5addc8ffe20f47749792ac1394f6235c => generated 611 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:49.649318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56243a49-1ffc-415f-b15d-f6bbeff650d9 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:49.649779 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56243a49-1ffc-415f-b15d-f6bbeff650d9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:49.649921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56243a49-1ffc-415f-b15d-f6bbeff650d9 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:49.651310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 212/428] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:21:49.669766 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cd82c-8821-44d1-9b9b-fd0a078084f3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:49.670047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:49.670349 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cd82c-8821-44d1-9b9b-fd0a078084f3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:49.670734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cd82c-8821-44d1-9b9b-fd0a078084f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:49.718380 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-660cd82c-8821-44d1-9b9b-fd0a078084f3 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:49.719616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-660cd82c-8821-44d1-9b9b-fd0a078084f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:49.748682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b0da5-de00-4331-9a7f-5ba2ed67e4e8 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:49.816561 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c0943-7fe1-45bf-95c9-2f026ae4e680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:49.818299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c0943-7fe1-45bf-95c9-2f026ae4e680 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:49.819054 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c0943-7fe1-45bf-95c9-2f026ae4e680 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:49.819957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c0943-7fe1-45bf-95c9-2f026ae4e680 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:49.855468 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 213/429] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:49.879279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660cd82c-8821-44d1-9b9b-fd0a078084f3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:49.966532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c0943-7fe1-45bf-95c9-2f026ae4e680 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:49.970935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c0943-7fe1-45bf-95c9-2f026ae4e680 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:50.008146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660cd82c-8821-44d1-9b9b-fd0a078084f3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:50.035337 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:50.052169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eabd7-546a-45e3-bbe6-bc1fba530ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:50.052991 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eabd7-546a-45e3-bbe6-bc1fba530ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:50.053303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eabd7-546a-45e3-bbe6-bc1fba530ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:50.053584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eabd7-546a-45e3-bbe6-bc1fba530ab6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:50.069532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89eabd7-546a-45e3-bbe6-bc1fba530ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f89eabd7-546a-45e3-bbe6-bc1fba530ab6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:50.082523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:50.083493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:50.084294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:50.084294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:50.084294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:50.084294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:50.084294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:50.084294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:50.084294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:50.084294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:50.084294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:50.084294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:50.084294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:50.084294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:50.084899 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 214/430] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:21:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:50.095158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eabd7-546a-45e3-bbe6-bc1fba530ab6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:50.113044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:50.172123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 217/431] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:21:50.177695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c0943-7fe1-45bf-95c9-2f026ae4e680 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:50.220455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eabd7-546a-45e3-bbe6-bc1fba530ab6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:50.259360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f88506-09f2-45c2-aede-dde7eaf0bccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:50.260037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f88506-09f2-45c2-aede-dde7eaf0bccd None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:50.260317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f88506-09f2-45c2-aede-dde7eaf0bccd None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:50.260585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f88506-09f2-45c2-aede-dde7eaf0bccd None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:50.261396 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f88506-09f2-45c2-aede-dde7eaf0bccd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:50.261704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f88506-09f2-45c2-aede-dde7eaf0bccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:50.263845 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f88506-09f2-45c2-aede-dde7eaf0bccd None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:50.275516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 215/432] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:50.281103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f88506-09f2-45c2-aede-dde7eaf0bccd None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:50.295795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c0943-7fe1-45bf-95c9-2f026ae4e680 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:50.341584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:50.349595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610184a-b756-44ea-89cb-83a923c56c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:50.350486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610184a-b756-44ea-89cb-83a923c56c1a None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:50.350772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610184a-b756-44ea-89cb-83a923c56c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:50.351043 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610184a-b756-44ea-89cb-83a923c56c1a None admin] PATH_INFO: `/v3/projects/d00b803196664866a4fd2eb970b1be0b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:50.352342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610184a-b756-44ea-89cb-83a923c56c1a None admin] RBAC: Authorizing `identity:get_project(project_id=d00b803196664866a4fd2eb970b1be0b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:50.353918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610184a-b756-44ea-89cb-83a923c56c1a None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:50.355855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 218/433] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:50 2022] GET /identity/v3/projects/d00b803196664866a4fd2eb970b1be0b => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:50.360530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3850e1-e2a9-4bbd-90ad-33185df6abf1 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:50.360530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3850e1-e2a9-4bbd-90ad-33185df6abf1 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:50.360530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3850e1-e2a9-4bbd-90ad-33185df6abf1 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:50.360530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 219/434] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:50.365286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3850e1-e2a9-4bbd-90ad-33185df6abf1 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:50.404922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:50.414282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f88506-09f2-45c2-aede-dde7eaf0bccd None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:50.451334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd7a8d-7ca5-4d27-9e4a-07a9b7be8391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:50.451994 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7a8d-7ca5-4d27-9e4a-07a9b7be8391 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:50.452281 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7a8d-7ca5-4d27-9e4a-07a9b7be8391 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:50.452558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7a8d-7ca5-4d27-9e4a-07a9b7be8391 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:50.485644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3850e1-e2a9-4bbd-90ad-33185df6abf1 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:50.524776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd38566-73bf-48c1-b03b-79a505d9a332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:50.525692 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd38566-73bf-48c1-b03b-79a505d9a332 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:50.525975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd38566-73bf-48c1-b03b-79a505d9a332 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:50.526251 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd38566-73bf-48c1-b03b-79a505d9a332 None admin] PATH_INFO: `/v3/groups/0a07d620279f4fcab3723e3460d7d2bc` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:50.527473 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd38566-73bf-48c1-b03b-79a505d9a332 None admin] RBAC: Authorizing `identity:get_group(group_id=0a07d620279f4fcab3723e3460d7d2bc)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:50.534849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd38566-73bf-48c1-b03b-79a505d9a332 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:50.536396 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd7a8d-7ca5-4d27-9e4a-07a9b7be8391 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:50.536573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 220/435] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:50 2022] GET /identity/v3/groups/0a07d620279f4fcab3723e3460d7d2bc => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:50.537885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e74fc5-71b0-44a0-a215-ea1caa433cac None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:50.538178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e74fc5-71b0-44a0-a215-ea1caa433cac None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:50.538483 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e74fc5-71b0-44a0-a215-ea1caa433cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:50.539447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd7a8d-7ca5-4d27-9e4a-07a9b7be8391 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:50.556721 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-98e74fc5-71b0-44a0-a215-ea1caa433cac None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:50.558021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-98e74fc5-71b0-44a0-a215-ea1caa433cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:50.602569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:50.645161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 221/436] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:50.645161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e74fc5-71b0-44a0-a215-ea1caa433cac None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:50.684222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 216/437] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:21:50.685685 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd7a8d-7ca5-4d27-9e4a-07a9b7be8391 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:50.702265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:50.801460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e74fc5-71b0-44a0-a215-ea1caa433cac None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:50.802939 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd7a8d-7ca5-4d27-9e4a-07a9b7be8391 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:50.842793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30d420-b466-472f-9003-07584e29a0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:50.843497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30d420-b466-472f-9003-07584e29a0be None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:50.843785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30d420-b466-472f-9003-07584e29a0be None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:50.844064 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30d420-b466-472f-9003-07584e29a0be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:50.844912 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30d420-b466-472f-9003-07584e29a0be None admin] RBAC: Adding query filter params (group.id=0a07d620279f4fcab3723e3460d7d2bc, role.id=2bdabfafa05e443e842fe34d5f03e2c9, scope.project.id=d00b803196664866a4fd2eb970b1be0b) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:50.845220 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be242334-72ad-4b39-b6a5-ed74767c9b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:50.845334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30d420-b466-472f-9003-07584e29a0be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:50.846036 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be242334-72ad-4b39-b6a5-ed74767c9b6e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:50.846316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be242334-72ad-4b39-b6a5-ed74767c9b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:50.846720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be242334-72ad-4b39-b6a5-ed74767c9b6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:50.846874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30d420-b466-472f-9003-07584e29a0be None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:50.854487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 222/438] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jun 7 08:21:50 2022] GET /identity/v3/role_assignments?group.id=0a07d620279f4fcab3723e3460d7d2bc&scope.project.id=d00b803196664866a4fd2eb970b1be0b&role.id=2bdabfafa05e443e842fe34d5f03e2c9 => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:50.860008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be242334-72ad-4b39-b6a5-ed74767c9b6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:50.862338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30d420-b466-472f-9003-07584e29a0be None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-be242334-72ad-4b39-b6a5-ed74767c9b6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:50.868010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:50.869745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:50.871282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:50.871282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:50.871282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:50.871282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:50.871282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:50.871282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:50.871282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:50.871282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:50.871282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:50.871282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:50.871282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:21:50.871282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:50.875864 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 217/439] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:50.876401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be242334-72ad-4b39-b6a5-ed74767c9b6e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:50.882258 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:51.015767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be242334-72ad-4b39-b6a5-ed74767c9b6e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:51.016285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:51.028950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30d420-b466-472f-9003-07584e29a0be None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:51.058622 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9275f-94a6-4b5b-bb3a-a5b1d5125fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:51.059338 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9275f-94a6-4b5b-bb3a-a5b1d5125fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:51.059629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9275f-94a6-4b5b-bb3a-a5b1d5125fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:51.059922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9275f-94a6-4b5b-bb3a-a5b1d5125fd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:51.060652 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9275f-94a6-4b5b-bb3a-a5b1d5125fd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:51.060955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9275f-94a6-4b5b-bb3a-a5b1d5125fd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:51.062385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9275f-94a6-4b5b-bb3a-a5b1d5125fd8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:51.072834 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 218/440] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:21:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:51.074236 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4b0ad-f395-4cf5-9c77-1b50709c1a31 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:51.074577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4b0ad-f395-4cf5-9c77-1b50709c1a31 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:51.074836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4b0ad-f395-4cf5-9c77-1b50709c1a31 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:51.077112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 219/441] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:51.079257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4b0ad-f395-4cf5-9c77-1b50709c1a31 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:51.082162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1cb4f-89ed-4dcf-b2f1-9dbd549b99d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:51.083041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1cb4f-89ed-4dcf-b2f1-9dbd549b99d1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:51.083322 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1cb4f-89ed-4dcf-b2f1-9dbd549b99d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:51.083933 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1cb4f-89ed-4dcf-b2f1-9dbd549b99d1 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:51.085202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1cb4f-89ed-4dcf-b2f1-9dbd549b99d1 None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:51.086666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1cb4f-89ed-4dcf-b2f1-9dbd549b99d1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:51.088407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 223/442] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:50 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:51.089814 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aa3e3-3ae3-4949-a689-c55ca6033b17 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:51.090092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aa3e3-3ae3-4949-a689-c55ca6033b17 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:51.090370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aa3e3-3ae3-4949-a689-c55ca6033b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:51.108881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-2d4aa3e3-3ae3-4949-a689-c55ca6033b17 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:51.110391 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-2d4aa3e3-3ae3-4949-a689-c55ca6033b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:51.171027 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:51.196143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 224/443] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:51.196915 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4aa3e3-3ae3-4949-a689-c55ca6033b17 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:51.222118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4b0ad-f395-4cf5-9c77-1b50709c1a31 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:51.268233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb71ab-04f9-451c-b587-6faec0d80f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:51.268906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb71ab-04f9-451c-b587-6faec0d80f15 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:51.269226 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb71ab-04f9-451c-b587-6faec0d80f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:51.269506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb71ab-04f9-451c-b587-6faec0d80f15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:51.313598 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:51.330857 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4aa3e3-3ae3-4949-a689-c55ca6033b17 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:51.371743 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a466d07-65f5-47c6-937a-661b565e1cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:51.375980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a466d07-65f5-47c6-937a-661b565e1cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:51.375980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a466d07-65f5-47c6-937a-661b565e1cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:51.375980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a466d07-65f5-47c6-937a-661b565e1cb1 None admin] PATH_INFO: `/v3/groups/0a07d620279f4fcab3723e3460d7d2bc` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:51.375980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a466d07-65f5-47c6-937a-661b565e1cb1 None admin] RBAC: Authorizing `identity:get_group(group_id=0a07d620279f4fcab3723e3460d7d2bc)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:51.376625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a466d07-65f5-47c6-937a-661b565e1cb1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:51.377403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb71ab-04f9-451c-b587-6faec0d80f15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:51.378363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 225/444] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:51 2022] GET /identity/v3/groups/0a07d620279f4fcab3723e3460d7d2bc => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:51.378865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb71ab-04f9-451c-b587-6faec0d80f15 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:51.379233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a466d07-65f5-47c6-937a-661b565e1cb1 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:51.463239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:51.497264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a466d07-65f5-47c6-937a-661b565e1cb1 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:51.506513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 220/445] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:21:51.508263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb71ab-04f9-451c-b587-6faec0d80f15 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:51.535040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e8a0c-904d-4fce-83c3-4a6409f5574c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:51.535839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e8a0c-904d-4fce-83c3-4a6409f5574c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:51.536119 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e8a0c-904d-4fce-83c3-4a6409f5574c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:51.536392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e8a0c-904d-4fce-83c3-4a6409f5574c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:51.550403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e8a0c-904d-4fce-83c3-4a6409f5574c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-921e8a0c-904d-4fce-83c3-4a6409f5574c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:51.558238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:51.559240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 226/446] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:51.560476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e8a0c-904d-4fce-83c3-4a6409f5574c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:51.581913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:51.622850 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb71ab-04f9-451c-b587-6faec0d80f15 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:51.658804 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a37712-dd4a-4178-a45f-e388cca53374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:51.659461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a37712-dd4a-4178-a45f-e388cca53374 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:51.659742 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a37712-dd4a-4178-a45f-e388cca53374 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:51.660013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a37712-dd4a-4178-a45f-e388cca53374 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:51.660838 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a37712-dd4a-4178-a45f-e388cca53374 None admin] RBAC: Adding query filter params (group.id=0a07d620279f4fcab3723e3460d7d2bc, role.id=5addc8ffe20f47749792ac1394f6235c, scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:51.661172 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a37712-dd4a-4178-a45f-e388cca53374 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:51.662660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a37712-dd4a-4178-a45f-e388cca53374 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:51.665670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e8a0c-904d-4fce-83c3-4a6409f5574c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:51.685119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 221/447] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jun 7 08:21:51 2022] GET /identity/v3/role_assignments?group.id=0a07d620279f4fcab3723e3460d7d2bc&scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b&role.id=5addc8ffe20f47749792ac1394f6235c => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:51.685119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a37712-dd4a-4178-a45f-e388cca53374 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:51.724306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8c5e1-2e89-40f8-a785-8087fe1b6973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:51.725440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8c5e1-2e89-40f8-a785-8087fe1b6973 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:51.725736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8c5e1-2e89-40f8-a785-8087fe1b6973 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:51.726017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8c5e1-2e89-40f8-a785-8087fe1b6973 None admin] PATH_INFO: `/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:51.727993 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8c5e1-2e89-40f8-a785-8087fe1b6973 None admin] RBAC: Authorizing `identity:get_project(project_id=a49ddfabaeee47e0ad65b7bf6470e03b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:51.729420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8c5e1-2e89-40f8-a785-8087fe1b6973 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:51.730720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:51.736371 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 227/448] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:51 2022] GET /identity/v3/projects/a49ddfabaeee47e0ad65b7bf6470e03b => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:51.739967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8c5e1-2e89-40f8-a785-8087fe1b6973 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:51.833264 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a37712-dd4a-4178-a45f-e388cca53374 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:51.854164 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:51.872755 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8c5e1-2e89-40f8-a785-8087fe1b6973 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:51.877634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c63e2c-d92f-45e6-9c5e-48f48da674de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:51.878319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c63e2c-d92f-45e6-9c5e-48f48da674de None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:51.878608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c63e2c-d92f-45e6-9c5e-48f48da674de None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:51.878880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c63e2c-d92f-45e6-9c5e-48f48da674de None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:51.879652 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c63e2c-d92f-45e6-9c5e-48f48da674de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:51.879953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c63e2c-d92f-45e6-9c5e-48f48da674de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:51.881382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c63e2c-d92f-45e6-9c5e-48f48da674de None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:51.894517 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 222/449] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:51.901808 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c63e2c-d92f-45e6-9c5e-48f48da674de None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:51.933455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bc200-28ab-4432-b169-cfc1d6a97d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:51.934976 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bc200-28ab-4432-b169-cfc1d6a97d2f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:51.935264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bc200-28ab-4432-b169-cfc1d6a97d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:51.935899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bc200-28ab-4432-b169-cfc1d6a97d2f None admin] PATH_INFO: `/v3/groups/0a07d620279f4fcab3723e3460d7d2bc` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:51.936854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bc200-28ab-4432-b169-cfc1d6a97d2f None admin] RBAC: Authorizing `identity:get_group(group_id=0a07d620279f4fcab3723e3460d7d2bc)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:51.938452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bc200-28ab-4432-b169-cfc1d6a97d2f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:51.940555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 228/450] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:51 2022] GET /identity/v3/groups/0a07d620279f4fcab3723e3460d7d2bc => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:51.946380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bc200-28ab-4432-b169-cfc1d6a97d2f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:52.033487 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:52.043958 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c63e2c-d92f-45e6-9c5e-48f48da674de None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:52.057671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bc200-28ab-4432-b169-cfc1d6a97d2f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:52.090197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4bc71-a340-4a06-91d8-40464879ff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:52.090821 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4bc71-a340-4a06-91d8-40464879ff24 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:52.091106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4bc71-a340-4a06-91d8-40464879ff24 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:52.091388 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4bc71-a340-4a06-91d8-40464879ff24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:52.104820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab1983-e59c-468d-b008-bcc3e271f683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:52.105520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab1983-e59c-468d-b008-bcc3e271f683 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:52.105803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab1983-e59c-468d-b008-bcc3e271f683 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:52.106079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab1983-e59c-468d-b008-bcc3e271f683 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:52.107049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab1983-e59c-468d-b008-bcc3e271f683 None admin] RBAC: Adding query filter params (group.id=0a07d620279f4fcab3723e3460d7d2bc, role.id=2bdabfafa05e443e842fe34d5f03e2c9, scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:52.107457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab1983-e59c-468d-b008-bcc3e271f683 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:52.109042 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab1983-e59c-468d-b008-bcc3e271f683 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:52.115364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 229/451] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jun 7 08:21:51 2022] GET /identity/v3/role_assignments?group.id=0a07d620279f4fcab3723e3460d7d2bc&scope.project.id=a49ddfabaeee47e0ad65b7bf6470e03b&role.id=2bdabfafa05e443e842fe34d5f03e2c9 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:52.179157 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4bc71-a340-4a06-91d8-40464879ff24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:52.180750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4bc71-a340-4a06-91d8-40464879ff24 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:52.289896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:52.304194 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 223/452] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:21:52.311710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab1983-e59c-468d-b008-bcc3e271f683 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:52.334345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:52.425701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab1983-e59c-468d-b008-bcc3e271f683 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:52.478063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359553c-c584-4ec2-ac07-5a0d1ac6c840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:52.478939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359553c-c584-4ec2-ac07-5a0d1ac6c840 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:52.479217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359553c-c584-4ec2-ac07-5a0d1ac6c840 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:52.479498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359553c-c584-4ec2-ac07-5a0d1ac6c840 None admin] PATH_INFO: `/v3/projects/ddc110e64f52436a913830733ab91055` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:52.480724 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359553c-c584-4ec2-ac07-5a0d1ac6c840 None admin] RBAC: Authorizing `identity:get_project(project_id=ddc110e64f52436a913830733ab91055)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:52.482752 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359553c-c584-4ec2-ac07-5a0d1ac6c840 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:52.484503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 230/453] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:52 2022] GET /identity/v3/projects/ddc110e64f52436a913830733ab91055 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:52.490101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4bc71-a340-4a06-91d8-40464879ff24 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:52.622881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4bc71-a340-4a06-91d8-40464879ff24 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:52.658753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6e6be-f40c-45bb-8085-b57a1485e8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:52.659605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6e6be-f40c-45bb-8085-b57a1485e8cd None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:52.659883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6e6be-f40c-45bb-8085-b57a1485e8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:52.660155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6e6be-f40c-45bb-8085-b57a1485e8cd None admin] PATH_INFO: `/v3/groups/7f125bf1255248b7a99fca7d6ba93a48` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:52.661337 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6e6be-f40c-45bb-8085-b57a1485e8cd None admin] RBAC: Authorizing `identity:get_group(group_id=7f125bf1255248b7a99fca7d6ba93a48)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:52.662788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6e6be-f40c-45bb-8085-b57a1485e8cd None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:52.664633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 224/454] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jun 7 08:21:52 2022] GET /identity/v3/groups/7f125bf1255248b7a99fca7d6ba93a48 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:52.671672 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359553c-c584-4ec2-ac07-5a0d1ac6c840 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:52.770542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:52.788058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359553c-c584-4ec2-ac07-5a0d1ac6c840 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:52.824530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcd407-db7a-4832-a758-df885191a4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:52.825219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcd407-db7a-4832-a758-df885191a4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:52.825510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcd407-db7a-4832-a758-df885191a4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:52.826099 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcd407-db7a-4832-a758-df885191a4a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:52.826622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcd407-db7a-4832-a758-df885191a4a1 None admin] RBAC: Adding query filter params (group.id=7f125bf1255248b7a99fca7d6ba93a48, role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=ddc110e64f52436a913830733ab91055) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:52.826932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcd407-db7a-4832-a758-df885191a4a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:52.828657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcd407-db7a-4832-a758-df885191a4a1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:52.835575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 231/455] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jun 7 08:21:52 2022] GET /identity/v3/role_assignments?group.id=7f125bf1255248b7a99fca7d6ba93a48&scope.project.id=ddc110e64f52436a913830733ab91055&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:55.828402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81700b3-918c-4a7e-86e2-82b1897535ae None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:55.829517 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81700b3-918c-4a7e-86e2-82b1897535ae None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:55.830041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81700b3-918c-4a7e-86e2-82b1897535ae None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:55.832234 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 225/456] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:55.837838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd764c9-9608-4686-8378-9f5044d634c9 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:55.838846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd764c9-9608-4686-8378-9f5044d634c9 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:55.839335 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd764c9-9608-4686-8378-9f5044d634c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:55.865958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1cd764c9-9608-4686-8378-9f5044d634c9 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:55.867575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1cd764c9-9608-4686-8378-9f5044d634c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:55.975052 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 232/457] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:55.990729 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c06c0-2dcf-4f3f-8e48-91194613cd66 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:55.991042 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c06c0-2dcf-4f3f-8e48-91194613cd66 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:55.991285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c06c0-2dcf-4f3f-8e48-91194613cd66 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:55.992710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 226/458] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:55.999013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1140889-1eff-40d5-b955-6d1368729c54 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:55.999322 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1140889-1eff-40d5-b955-6d1368729c54 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:55.999564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1140889-1eff-40d5-b955-6d1368729c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:56.011377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c06c0-2dcf-4f3f-8e48-91194613cd66 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:56.021686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e1140889-1eff-40d5-b955-6d1368729c54 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:56.022988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e1140889-1eff-40d5-b955-6d1368729c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:56.113997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 233/459] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:21:56.142247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1140889-1eff-40d5-b955-6d1368729c54 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:56.155081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:56.183728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:56.262001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c06c0-2dcf-4f3f-8e48-91194613cd66 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:56.308831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1140889-1eff-40d5-b955-6d1368729c54 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:56.311503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac821255-7414-4014-9242-67c0ee605773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:56.313341 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac821255-7414-4014-9242-67c0ee605773 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:56.313644 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac821255-7414-4014-9242-67c0ee605773 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:56.313921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac821255-7414-4014-9242-67c0ee605773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:56.334107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac821255-7414-4014-9242-67c0ee605773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:56.336000 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac821255-7414-4014-9242-67c0ee605773 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-ac821255-7414-4014-9242-67c0ee605773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:21:56.361437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:56.362366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:56.363196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:21:56.363196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:21:56.363196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:56.363196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:56.364818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd41760-2edb-410e-a8d3-628ef1dbe4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:56.370028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 227/460] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:56.370203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd41760-2edb-410e-a8d3-628ef1dbe4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:56.370203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd41760-2edb-410e-a8d3-628ef1dbe4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:56.370203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd41760-2edb-410e-a8d3-628ef1dbe4a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:56.370715 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f97b6-fb0a-4289-8695-e0d18718b952 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:56.371025 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f97b6-fb0a-4289-8695-e0d18718b952 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:56.371305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f97b6-fb0a-4289-8695-e0d18718b952 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:56.374213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 228/461] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:56.375027 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f97b6-fb0a-4289-8695-e0d18718b952 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:56.391490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd41760-2edb-410e-a8d3-628ef1dbe4a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:56.394016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd41760-2edb-410e-a8d3-628ef1dbe4a3 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-8cd41760-2edb-410e-a8d3-628ef1dbe4a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:21:56.417839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:56.418823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:56.419662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:21:56.419662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:21:56.419662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:56.419662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:56.422173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 234/462] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:56.428287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bffbea-d7fe-4964-8855-c5d929cfc0fd None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:56.428577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bffbea-d7fe-4964-8855-c5d929cfc0fd None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:56.428891 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bffbea-d7fe-4964-8855-c5d929cfc0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:56.447165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e7bffbea-d7fe-4964-8855-c5d929cfc0fd None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:56.448386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e7bffbea-d7fe-4964-8855-c5d929cfc0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:56.513394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f97b6-fb0a-4289-8695-e0d18718b952 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:56.526505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:56.530232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:56.543726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 235/463] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:56.545553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bddd6d-b804-464e-a4f8-caceb5fee33f None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:56.545841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bddd6d-b804-464e-a4f8-caceb5fee33f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:56.546116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bddd6d-b804-464e-a4f8-caceb5fee33f None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:56.552881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 236/464] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:56.552881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bddd6d-b804-464e-a4f8-caceb5fee33f None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:56.574671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a50e9-8309-4ca8-8c7a-fba2f81ac426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:56.576581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a50e9-8309-4ca8-8c7a-fba2f81ac426 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:56.576581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a50e9-8309-4ca8-8c7a-fba2f81ac426 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:56.576581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a50e9-8309-4ca8-8c7a-fba2f81ac426 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:56.576581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a50e9-8309-4ca8-8c7a-fba2f81ac426 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:56.576581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a50e9-8309-4ca8-8c7a-fba2f81ac426 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:56.582450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a50e9-8309-4ca8-8c7a-fba2f81ac426 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:56.608787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 229/465] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:56.608787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10906f-9b87-4628-b379-83b176a24679 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:56.609180 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10906f-9b87-4628-b379-83b176a24679 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:56.610117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10906f-9b87-4628-b379-83b176a24679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:56.633275 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-eb10906f-9b87-4628-b379-83b176a24679 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:56.634522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-eb10906f-9b87-4628-b379-83b176a24679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:56.691338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bddd6d-b804-464e-a4f8-caceb5fee33f None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:56.741112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 230/466] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:56.741112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10906f-9b87-4628-b379-83b176a24679 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:56.768102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2922d-d85c-42f3-920c-9ecc180fad49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:56.769135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2922d-d85c-42f3-920c-9ecc180fad49 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:56.769433 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2922d-d85c-42f3-920c-9ecc180fad49 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:56.769836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2922d-d85c-42f3-920c-9ecc180fad49 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:56.771080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2922d-d85c-42f3-920c-9ecc180fad49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:56.771399 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2922d-d85c-42f3-920c-9ecc180fad49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:56.773317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2922d-d85c-42f3-920c-9ecc180fad49 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:56.789683 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 237/467] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:56.793252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2922d-d85c-42f3-920c-9ecc180fad49 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:56.857257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10906f-9b87-4628-b379-83b176a24679 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:56.857879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:56.859228 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:56.908717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbd368-9654-4aa1-a75b-a2a65e7e5891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:56.909835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbd368-9654-4aa1-a75b-a2a65e7e5891 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:56.910277 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbd368-9654-4aa1-a75b-a2a65e7e5891 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:56.910736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbd368-9654-4aa1-a75b-a2a65e7e5891 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:56.924813 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbd368-9654-4aa1-a75b-a2a65e7e5891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:56.926316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbd368-9654-4aa1-a75b-a2a65e7e5891 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:56.934046 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2922d-d85c-42f3-920c-9ecc180fad49 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-3dfbd368-9654-4aa1-a75b-a2a65e7e5891 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:21:56.944763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:56.945855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:56.946769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:21:56.946769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:21:56.946769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:56.946769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:56.946769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 231/468] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:56.947095 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbd368-9654-4aa1-a75b-a2a65e7e5891 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:56.980306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafc9af-4333-433c-a3cf-dfb35f2e96e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:56.981624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafc9af-4333-433c-a3cf-dfb35f2e96e8 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:56.981837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafc9af-4333-433c-a3cf-dfb35f2e96e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:56.982136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafc9af-4333-433c-a3cf-dfb35f2e96e8 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:56.982932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafc9af-4333-433c-a3cf-dfb35f2e96e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:56.984448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafc9af-4333-433c-a3cf-dfb35f2e96e8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:57.009527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 238/469] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:21:56 2022] POST /identity/v3/users => generated 246 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:57.010224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafc9af-4333-433c-a3cf-dfb35f2e96e8 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:57.104473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbd368-9654-4aa1-a75b-a2a65e7e5891 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:57.173849 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85311f0-2ddb-4eee-a49d-9c98515740a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:57.175157 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85311f0-2ddb-4eee-a49d-9c98515740a0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:57.176212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85311f0-2ddb-4eee-a49d-9c98515740a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:57.176686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85311f0-2ddb-4eee-a49d-9c98515740a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:57.183766 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafc9af-4333-433c-a3cf-dfb35f2e96e8 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:57.194522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85311f0-2ddb-4eee-a49d-9c98515740a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:57.196362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85311f0-2ddb-4eee-a49d-9c98515740a0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:57.208930 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-b85311f0-2ddb-4eee-a49d-9c98515740a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:21:57.229443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:57.230454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:57.231391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:21:57.231391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:21:57.231391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:57.231391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:57.234241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 232/470] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:57.235841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85311f0-2ddb-4eee-a49d-9c98515740a0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:57.253282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:57.262198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804d573-acb9-4f06-be74-55180f777ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:57.264029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804d573-acb9-4f06-be74-55180f777ac2 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:57.264817 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804d573-acb9-4f06-be74-55180f777ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:57.265800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804d573-acb9-4f06-be74-55180f777ac2 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:57.267407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804d573-acb9-4f06-be74-55180f777ac2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:57.270735 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804d573-acb9-4f06-be74-55180f777ac2 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:57.303191 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 239/471] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:21:57 2022] POST /identity/v3/users => generated 248 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:57.304109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804d573-acb9-4f06-be74-55180f777ac2 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:57.428526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85311f0-2ddb-4eee-a49d-9c98515740a0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:57.483669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804d573-acb9-4f06-be74-55180f777ac2 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:57.484100 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee5e94-0c75-4f5a-8039-3bdf1c314f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:57.484681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee5e94-0c75-4f5a-8039-3bdf1c314f8d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:57.484999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee5e94-0c75-4f5a-8039-3bdf1c314f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:57.485492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee5e94-0c75-4f5a-8039-3bdf1c314f8d None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:57.486620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee5e94-0c75-4f5a-8039-3bdf1c314f8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:57.487158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee5e94-0c75-4f5a-8039-3bdf1c314f8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:57.489678 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee5e94-0c75-4f5a-8039-3bdf1c314f8d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:57.517236 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 233/472] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:57.520406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72842fd-46b3-4311-bde0-53710ec4725a None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:57.521353 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72842fd-46b3-4311-bde0-53710ec4725a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:57.522107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72842fd-46b3-4311-bde0-53710ec4725a None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:57.525511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 234/473] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:57.527680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72842fd-46b3-4311-bde0-53710ec4725a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:57.556068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6715614-808d-48d9-96f3-7184086943d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:57.556759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6715614-808d-48d9-96f3-7184086943d1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:57.557044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6715614-808d-48d9-96f3-7184086943d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:57.557355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6715614-808d-48d9-96f3-7184086943d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:57.558092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6715614-808d-48d9-96f3-7184086943d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:57.558403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6715614-808d-48d9-96f3-7184086943d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:57.559884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6715614-808d-48d9-96f3-7184086943d1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:57.577174 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 240/474] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:57.577557 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94c43e-5f0e-4f99-a184-8458b3dd5e18 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:57.577851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94c43e-5f0e-4f99-a184-8458b3dd5e18 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:57.578379 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94c43e-5f0e-4f99-a184-8458b3dd5e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:57.599010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-2a94c43e-5f0e-4f99-a184-8458b3dd5e18 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:57.601222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-2a94c43e-5f0e-4f99-a184-8458b3dd5e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:57.636449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:57.661502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:57.719633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72842fd-46b3-4311-bde0-53710ec4725a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:57.733165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 241/475] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:21:57.733165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94c43e-5f0e-4f99-a184-8458b3dd5e18 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:57.789945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe1c73-031f-46d3-a24a-6fb0f60d9500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:57.792736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe1c73-031f-46d3-a24a-6fb0f60d9500 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:57.794604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe1c73-031f-46d3-a24a-6fb0f60d9500 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:57.795408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe1c73-031f-46d3-a24a-6fb0f60d9500 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:57.796685 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe1c73-031f-46d3-a24a-6fb0f60d9500 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:57.798498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe1c73-031f-46d3-a24a-6fb0f60d9500 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:57.837751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 235/476] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:21:57 2022] POST /identity/v3/users => generated 248 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:57.838227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe1c73-031f-46d3-a24a-6fb0f60d9500 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:57.967813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94c43e-5f0e-4f99-a184-8458b3dd5e18 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:58.026866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe1c73-031f-46d3-a24a-6fb0f60d9500 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:58.058418 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:58.079872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e947d4a-ded5-452b-9f8e-aef84df80733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:58.080815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfba71-cbf4-4898-a588-495d36454ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:58.081713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfba71-cbf4-4898-a588-495d36454ed5 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:58.082707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e947d4a-ded5-452b-9f8e-aef84df80733 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:58.083212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfba71-cbf4-4898-a588-495d36454ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:58.083732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfba71-cbf4-4898-a588-495d36454ed5 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:58.084186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e947d4a-ded5-452b-9f8e-aef84df80733 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:58.085125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfba71-cbf4-4898-a588-495d36454ed5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:58.085643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e947d4a-ded5-452b-9f8e-aef84df80733 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:58.088028 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfba71-cbf4-4898-a588-495d36454ed5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:58.088495 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e947d4a-ded5-452b-9f8e-aef84df80733 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:58.090107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e947d4a-ded5-452b-9f8e-aef84df80733 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:58.107190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.sql.core [None req-2e947d4a-ded5-452b-9f8e-aef84df80733 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 08:21:58.107190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 08:21:58.107190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [parameters: {'id': 'e7d1911c788447c69dc4189a8f262155', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 08:21:58.107190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80228) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-2e947d4a-ded5-452b-9f8e-aef84df80733 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application conn.query(q) Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application result.read() Jun 07 08:21:58.132085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return ref.to_dict() Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 08:21:58.133232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application next(self.gen) Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application yield resource Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application next(self.gen) Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise self.value Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application session.commit() Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 08:21:58.134357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.session.flush() Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application compat.raise_( Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 08:21:58.135331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 08:21:58.136202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application conn.query(q) Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application result.read() Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 08:21:58.137003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [parameters: {'id': 'e7d1911c788447c69dc4189a8f262155', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:58.137881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 08:21:58.138809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:21:58.144057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 242/477] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:21:57 2022] POST /identity/v3/users => generated 249 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:58.145940 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 236/478] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Jun 7 08:21:57 2022] POST /identity/v3/roles => generated 144 bytes in 308 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:58.154626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfba71-cbf4-4898-a588-495d36454ed5 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:58.177555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:58.317422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfba71-cbf4-4898-a588-495d36454ed5 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:58.359258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576a937-7ea7-41a8-8622-4bc614967c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:58.360098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576a937-7ea7-41a8-8622-4bc614967c3c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:58.360380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576a937-7ea7-41a8-8622-4bc614967c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:58.360654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576a937-7ea7-41a8-8622-4bc614967c3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:58.373901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576a937-7ea7-41a8-8622-4bc614967c3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-8576a937-7ea7-41a8-8622-4bc614967c3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:58.382402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:58.383361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:58.384138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:58.384138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:58.384138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:58.384138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:58.384138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:58.384138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:58.384138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:58.384138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:58.384138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:58.384138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:58.384138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:21:58.384138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:21:58.388636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 243/479] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:21:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:58.390071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e947d4a-ded5-452b-9f8e-aef84df80733 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:58.524789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e947d4a-ded5-452b-9f8e-aef84df80733 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:58.569779 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb13fc-6f56-4843-bcb6-2d71cbc2f494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:58.570525 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb13fc-6f56-4843-bcb6-2d71cbc2f494 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:58.570856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb13fc-6f56-4843-bcb6-2d71cbc2f494 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:58.571119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb13fc-6f56-4843-bcb6-2d71cbc2f494 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:58.571896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb13fc-6f56-4843-bcb6-2d71cbc2f494 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:58.572379 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb13fc-6f56-4843-bcb6-2d71cbc2f494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:58.573990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb13fc-6f56-4843-bcb6-2d71cbc2f494 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:58.584100 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 237/480] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:59.922959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52edc7e-7855-47c9-adca-689eb04dc857 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:59.923998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52edc7e-7855-47c9-adca-689eb04dc857 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:59.924477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52edc7e-7855-47c9-adca-689eb04dc857 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:59.926310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 244/481] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:21:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:59.939891 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a2da9-6766-41da-a30d-993ae5fd6c98 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:59.940867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a2da9-6766-41da-a30d-993ae5fd6c98 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:59.941394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a2da9-6766-41da-a30d-993ae5fd6c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:59.966863 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-765a2da9-6766-41da-a30d-993ae5fd6c98 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:59.968429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-765a2da9-6766-41da-a30d-993ae5fd6c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:00.025182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:00.061005 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 238/482] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:00.096665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52edc7e-7855-47c9-adca-689eb04dc857 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:00.203329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:00.250391 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52edc7e-7855-47c9-adca-689eb04dc857 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:00.298694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870bb95-b7d3-4fb5-8bdd-1880e5a05259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:00.299883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870bb95-b7d3-4fb5-8bdd-1880e5a05259 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:00.300384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870bb95-b7d3-4fb5-8bdd-1880e5a05259 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:00.300862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870bb95-b7d3-4fb5-8bdd-1880e5a05259 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:00.318352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8870bb95-b7d3-4fb5-8bdd-1880e5a05259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-8870bb95-b7d3-4fb5-8bdd-1880e5a05259 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:00.326913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:00.327920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:00.328996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:00.328996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:00.328996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:00.328996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:00.328996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:00.328996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:00.328996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:00.328996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:00.328996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:00.328996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:00.328996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:00.328996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:00.331363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 245/483] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:00.336681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a2da9-6766-41da-a30d-993ae5fd6c98 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:00.457031 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a2da9-6766-41da-a30d-993ae5fd6c98 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:00.494887 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7e3cc-c87f-458a-a022-160c6b88770e None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:00.495235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7e3cc-c87f-458a-a022-160c6b88770e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:00.495432 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7e3cc-c87f-458a-a022-160c6b88770e None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:00.497363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 246/484] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:00.502681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48e9d2-55e0-4030-8612-3be08ae6d684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:00.503478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1e2f7-cfd7-4fa2-814c-7990b52a1e8a None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:00.503771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1e2f7-cfd7-4fa2-814c-7990b52a1e8a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:00.504049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1e2f7-cfd7-4fa2-814c-7990b52a1e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:00.504420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48e9d2-55e0-4030-8612-3be08ae6d684 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:00.504904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48e9d2-55e0-4030-8612-3be08ae6d684 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:00.505417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48e9d2-55e0-4030-8612-3be08ae6d684 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:00.506408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48e9d2-55e0-4030-8612-3be08ae6d684 None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:00.506932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48e9d2-55e0-4030-8612-3be08ae6d684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:00.508572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48e9d2-55e0-4030-8612-3be08ae6d684 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:00.519415 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 239/485] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:00.525180 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48e9d2-55e0-4030-8612-3be08ae6d684 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:00.545935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d4c1e2f7-cfd7-4fa2-814c-7990b52a1e8a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:00.547157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d4c1e2f7-cfd7-4fa2-814c-7990b52a1e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:00.592395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:00.659224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 247/486] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:00.701052 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1e2f7-cfd7-4fa2-814c-7990b52a1e8a None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:00.718273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48e9d2-55e0-4030-8612-3be08ae6d684 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:00.767694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5774fa-8cc5-498f-b6cf-f0c1eeb52844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:00.768503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5774fa-8cc5-498f-b6cf-f0c1eeb52844 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:00.768792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5774fa-8cc5-498f-b6cf-f0c1eeb52844 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:00.769065 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5774fa-8cc5-498f-b6cf-f0c1eeb52844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:00.794817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5774fa-8cc5-498f-b6cf-f0c1eeb52844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:00.795850 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:00.796314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5774fa-8cc5-498f-b6cf-f0c1eeb52844 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-0e5774fa-8cc5-498f-b6cf-f0c1eeb52844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:00.834250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:00.835189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:00.836054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:00.836054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:00.836054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:00.836054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:00.836054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 240/487] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:00.841016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1e2f7-cfd7-4fa2-814c-7990b52a1e8a None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:00.843484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5774fa-8cc5-498f-b6cf-f0c1eeb52844 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:00.888572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3615fc-b0f3-4b5d-9fa1-4cad72b0f80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:00.889808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3615fc-b0f3-4b5d-9fa1-4cad72b0f80d None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:00.890512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3615fc-b0f3-4b5d-9fa1-4cad72b0f80d None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:00.891206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3615fc-b0f3-4b5d-9fa1-4cad72b0f80d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:00.931622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3615fc-b0f3-4b5d-9fa1-4cad72b0f80d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-1e3615fc-b0f3-4b5d-9fa1-4cad72b0f80d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:00.951396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:00.952364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:00.953913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:00.953913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:00.953913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:00.953913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:00.953913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:00.953913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:00.953913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:00.953913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:00.953913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:00.953913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:00.953913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:00.953913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:00.953913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 248/488] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:00.960362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3615fc-b0f3-4b5d-9fa1-4cad72b0f80d None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:00.964351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:00.980763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5774fa-8cc5-498f-b6cf-f0c1eeb52844 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:01.019930 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296fd5c6-470c-4c8a-8e5b-0cba130a6d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:01.020599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fd5c6-470c-4c8a-8e5b-0cba130a6d35 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.020886 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fd5c6-470c-4c8a-8e5b-0cba130a6d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.021198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fd5c6-470c-4c8a-8e5b-0cba130a6d35 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.021924 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296fd5c6-470c-4c8a-8e5b-0cba130a6d35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:01.022244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296fd5c6-470c-4c8a-8e5b-0cba130a6d35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:01.023597 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296fd5c6-470c-4c8a-8e5b-0cba130a6d35 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:01.037076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 241/489] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:01.038495 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7da20c-7375-4872-b763-d16f7856b532 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.038783 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7da20c-7375-4872-b763-d16f7856b532 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.039053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7da20c-7375-4872-b763-d16f7856b532 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.040428 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 242/490] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:01.045887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877269e-17a0-41f0-a98f-190ec224516d None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.046171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877269e-17a0-41f0-a98f-190ec224516d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.046459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877269e-17a0-41f0-a98f-190ec224516d None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.047987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 243/491] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:01.048504 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877269e-17a0-41f0-a98f-190ec224516d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:01.097513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3615fc-b0f3-4b5d-9fa1-4cad72b0f80d None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:01.119143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:01.146094 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34a83c-ce85-4754-a39d-5a485d3f0b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:01.146772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34a83c-ce85-4754-a39d-5a485d3f0b6b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.147062 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34a83c-ce85-4754-a39d-5a485d3f0b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.147730 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34a83c-ce85-4754-a39d-5a485d3f0b6b None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.148093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34a83c-ce85-4754-a39d-5a485d3f0b6b None admin] RBAC: Adding query filter params (name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:01.148401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34a83c-ce85-4754-a39d-5a485d3f0b6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:01.149956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34a83c-ce85-4754-a39d-5a485d3f0b6b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:01.159752 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 249/492] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:01.160856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dae70-8dc2-4b08-958c-593f4e71dfd1 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.161184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dae70-8dc2-4b08-958c-593f4e71dfd1 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.161468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dae70-8dc2-4b08-958c-593f4e71dfd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.163038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877269e-17a0-41f0-a98f-190ec224516d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:01.178396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-551dae70-8dc2-4b08-958c-593f4e71dfd1 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:01.179629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-551dae70-8dc2-4b08-958c-593f4e71dfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:01.203593 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dcc87-7e83-4f17-a4f7-a524547083fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:01.204408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dcc87-7e83-4f17-a4f7-a524547083fd None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.204691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dcc87-7e83-4f17-a4f7-a524547083fd None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.204963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dcc87-7e83-4f17-a4f7-a524547083fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.213313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2dcc87-7e83-4f17-a4f7-a524547083fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:01.215320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2dcc87-7e83-4f17-a4f7-a524547083fd None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-4a2dcc87-7e83-4f17-a4f7-a524547083fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:01.232822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:01.233817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:01.234835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:01.234835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:01.234835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:01.234835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:01.234835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:01.234835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:01.234835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:01.234835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:01.234835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:22:01.234835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:01.234835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 244/493] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jun 7 08:22:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:01.236053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45da99-4b16-417b-b9f2-e746d6f8192c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.236053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45da99-4b16-417b-b9f2-e746d6f8192c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.236229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45da99-4b16-417b-b9f2-e746d6f8192c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.255144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7f45da99-4b16-417b-b9f2-e746d6f8192c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:01.256348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7f45da99-4b16-417b-b9f2-e746d6f8192c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:01.265131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:01.277405 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 250/494] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:01.278515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dae70-8dc2-4b08-958c-593f4e71dfd1 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:01.337368 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 245/495] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:01.341509 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45da99-4b16-417b-b9f2-e746d6f8192c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:01.410018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dae70-8dc2-4b08-958c-593f4e71dfd1 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:01.468595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45da99-4b16-417b-b9f2-e746d6f8192c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:01.508867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f997de-6af3-4dd1-8a4f-d272528dd506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:01.509506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f997de-6af3-4dd1-8a4f-d272528dd506 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.509790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f997de-6af3-4dd1-8a4f-d272528dd506 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.510066 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f997de-6af3-4dd1-8a4f-d272528dd506 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.510922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f997de-6af3-4dd1-8a4f-d272528dd506 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:01.511235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f997de-6af3-4dd1-8a4f-d272528dd506 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:01.513595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f997de-6af3-4dd1-8a4f-d272528dd506 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:01.552154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:01.556251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:01.563920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 246/496] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jun 7 08:22:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:01.565006 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f997de-6af3-4dd1-8a4f-d272528dd506 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:01.589855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fafed9-117a-4692-8161-21c4565f396b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:01.591267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fafed9-117a-4692-8161-21c4565f396b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.591544 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fafed9-117a-4692-8161-21c4565f396b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.591938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fafed9-117a-4692-8161-21c4565f396b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.602426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fafed9-117a-4692-8161-21c4565f396b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:01.604056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fafed9-117a-4692-8161-21c4565f396b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-94fafed9-117a-4692-8161-21c4565f396b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:01.622796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:01.623737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:01.624602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:01.624602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:01.624602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:01.624602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:01.624602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 251/497] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:01.625398 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fafed9-117a-4692-8161-21c4565f396b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:01.683446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f997de-6af3-4dd1-8a4f-d272528dd506 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:01.723867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5efb5b-1783-43a8-bb15-8b77297a5276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:01.724675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5efb5b-1783-43a8-bb15-8b77297a5276 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.724960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5efb5b-1783-43a8-bb15-8b77297a5276 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.725282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5efb5b-1783-43a8-bb15-8b77297a5276 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.733321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fafed9-117a-4692-8161-21c4565f396b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:01.737953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5efb5b-1783-43a8-bb15-8b77297a5276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-fb5efb5b-1783-43a8-bb15-8b77297a5276 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:01.746074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:01.747581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:01.748360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:01.748360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:01.748360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:01.748360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:01.748360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:01.748360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:01.748360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:01.748360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:01.748360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:01.748360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:01.748360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:01.748360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:01.748360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 247/498] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:01.749034 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5efb5b-1783-43a8-bb15-8b77297a5276 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:01.786999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec7146-329f-43f8-b12d-9207a1e80160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:01.787727 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec7146-329f-43f8-b12d-9207a1e80160 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.788020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec7146-329f-43f8-b12d-9207a1e80160 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.788303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec7146-329f-43f8-b12d-9207a1e80160 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.804187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec7146-329f-43f8-b12d-9207a1e80160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-a6ec7146-329f-43f8-b12d-9207a1e80160 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:01.818297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:01.819426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:01.820232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:01.820232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:01.820232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:01.820232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:01.820232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:01.820232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:01.820232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:01.820232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:01.820232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:01.820232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:01.820232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:01.820232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:01.827977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 252/499] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:01.829632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec7146-329f-43f8-b12d-9207a1e80160 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:01.834827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:01.864071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:01.882542 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5efb5b-1783-43a8-bb15-8b77297a5276 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:01.922450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17ecf8-a2b3-4d5c-bfb9-88c2a01be608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:01.923099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17ecf8-a2b3-4d5c-bfb9-88c2a01be608 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.923379 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17ecf8-a2b3-4d5c-bfb9-88c2a01be608 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.923653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17ecf8-a2b3-4d5c-bfb9-88c2a01be608 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.948399 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec7146-329f-43f8-b12d-9207a1e80160 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:01.984547 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ece7a4-ef82-484f-a1eb-0af49e23dc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:01.985249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ece7a4-ef82-484f-a1eb-0af49e23dc62 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.985536 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ece7a4-ef82-484f-a1eb-0af49e23dc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.985813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ece7a4-ef82-484f-a1eb-0af49e23dc62 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.986537 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ece7a4-ef82-484f-a1eb-0af49e23dc62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:01.986843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ece7a4-ef82-484f-a1eb-0af49e23dc62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:01.988223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ece7a4-ef82-484f-a1eb-0af49e23dc62 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:01.995552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17ecf8-a2b3-4d5c-bfb9-88c2a01be608 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:01.997619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17ecf8-a2b3-4d5c-bfb9-88c2a01be608 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:02.001646 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 253/500] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:02.002378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ece7a4-ef82-484f-a1eb-0af49e23dc62 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:02.078863 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:02.094238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:02.113920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ece7a4-ef82-484f-a1eb-0af49e23dc62 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:02.124769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 248/501] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:22:02.127051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03090329-d558-47c6-b3b3-15be027d9a0e None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:02.127492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03090329-d558-47c6-b3b3-15be027d9a0e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:02.127904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03090329-d558-47c6-b3b3-15be027d9a0e None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:02.129449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 249/502] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:02.130647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03090329-d558-47c6-b3b3-15be027d9a0e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:02.151694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd1871-206e-43ac-b55f-36de929d0280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:02.152454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd1871-206e-43ac-b55f-36de929d0280 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:02.152780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd1871-206e-43ac-b55f-36de929d0280 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:02.153306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd1871-206e-43ac-b55f-36de929d0280 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:02.154454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd1871-206e-43ac-b55f-36de929d0280 None admin] RBAC: Adding query filter params (name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:02.154803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd1871-206e-43ac-b55f-36de929d0280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:02.156207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd1871-206e-43ac-b55f-36de929d0280 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:02.164867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 254/503] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:02.165725 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd1871-206e-43ac-b55f-36de929d0280 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:02.238827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03090329-d558-47c6-b3b3-15be027d9a0e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:02.268658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd1871-206e-43ac-b55f-36de929d0280 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:02.274485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cadb9f8-f313-4b1a-95aa-f19e81a32f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:02.275161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cadb9f8-f313-4b1a-95aa-f19e81a32f1b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:02.275450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cadb9f8-f313-4b1a-95aa-f19e81a32f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:02.275729 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cadb9f8-f313-4b1a-95aa-f19e81a32f1b None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:02.277623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cadb9f8-f313-4b1a-95aa-f19e81a32f1b None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:02.277623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cadb9f8-f313-4b1a-95aa-f19e81a32f1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:02.278234 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cadb9f8-f313-4b1a-95aa-f19e81a32f1b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:02.286832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 250/504] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:02.287599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cadb9f8-f313-4b1a-95aa-f19e81a32f1b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:02.303306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794856c-7561-4081-9df0-07e95bc23f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:02.304102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794856c-7561-4081-9df0-07e95bc23f4b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:02.304384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794856c-7561-4081-9df0-07e95bc23f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:02.304654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794856c-7561-4081-9df0-07e95bc23f4b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:02.313789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794856c-7561-4081-9df0-07e95bc23f4b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:02.314738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:02.315234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794856c-7561-4081-9df0-07e95bc23f4b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-4794856c-7561-4081-9df0-07e95bc23f4b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:02.331484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:02.332632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:02.334127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:02.334127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:02.334127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:02.334127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:02.334127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:02.334127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:02.334127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:02.334127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:02.334127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:22:02.334127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:02.334127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 255/505] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:02.334127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe49bd-27b8-4ccb-bba0-b7a1836cca37 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:02.334127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe49bd-27b8-4ccb-bba0-b7a1836cca37 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:02.334808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe49bd-27b8-4ccb-bba0-b7a1836cca37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:02.346277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:02.351940 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-7dbe49bd-27b8-4ccb-bba0-b7a1836cca37 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:02.353198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-7dbe49bd-27b8-4ccb-bba0-b7a1836cca37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:02.394739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cadb9f8-f313-4b1a-95aa-f19e81a32f1b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:02.422633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 256/506] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:02.423496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe49bd-27b8-4ccb-bba0-b7a1836cca37 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:02.440790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8932c3f-2e4d-43e5-887e-229465f6022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:02.441634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8932c3f-2e4d-43e5-887e-229465f6022a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:02.441919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8932c3f-2e4d-43e5-887e-229465f6022a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:02.442197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8932c3f-2e4d-43e5-887e-229465f6022a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:02.453465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8932c3f-2e4d-43e5-887e-229465f6022a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:02.454908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8932c3f-2e4d-43e5-887e-229465f6022a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-c8932c3f-2e4d-43e5-887e-229465f6022a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:02.472599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:02.473671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:02.474702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:02.474702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:02.474702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:02.474702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:02.474702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 251/507] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:02.474702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8932c3f-2e4d-43e5-887e-229465f6022a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:02.528784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe49bd-27b8-4ccb-bba0-b7a1836cca37 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:02.553289 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:02.565553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105024a6-cb6d-4a40-9405-1c7242576796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:02.566355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105024a6-cb6d-4a40-9405-1c7242576796 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:02.566643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105024a6-cb6d-4a40-9405-1c7242576796 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:02.566921 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105024a6-cb6d-4a40-9405-1c7242576796 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:02.569825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:02.578505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105024a6-cb6d-4a40-9405-1c7242576796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:02.579975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105024a6-cb6d-4a40-9405-1c7242576796 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:02.580818 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8932c3f-2e4d-43e5-887e-229465f6022a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-105024a6-cb6d-4a40-9405-1c7242576796 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:02.598425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:02.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:02.600157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:02.600157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:02.600157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:02.600157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:02.600157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 257/508] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:02.600427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105024a6-cb6d-4a40-9405-1c7242576796 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:02.618237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5205adb-db84-4aab-95fe-5232a8468ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:02.619030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5205adb-db84-4aab-95fe-5232a8468ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:02.619325 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5205adb-db84-4aab-95fe-5232a8468ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:02.619598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5205adb-db84-4aab-95fe-5232a8468ed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:02.630578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5205adb-db84-4aab-95fe-5232a8468ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:02.632067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5205adb-db84-4aab-95fe-5232a8468ed5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-e5205adb-db84-4aab-95fe-5232a8468ed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:02.650503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:02.651569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:02.653708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:02.653708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:02.653708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:02.653708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:02.653708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 252/509] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:02.653708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5205adb-db84-4aab-95fe-5232a8468ed5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:02.709327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105024a6-cb6d-4a40-9405-1c7242576796 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:02.746564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7884f79-c267-4116-a4cb-b68db3449e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:02.747306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7884f79-c267-4116-a4cb-b68db3449e54 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:02.747770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7884f79-c267-4116-a4cb-b68db3449e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:02.748052 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7884f79-c267-4116-a4cb-b68db3449e54 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:02.748914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7884f79-c267-4116-a4cb-b68db3449e54 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:02.749262 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7884f79-c267-4116-a4cb-b68db3449e54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:02.750713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7884f79-c267-4116-a4cb-b68db3449e54 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:02.760075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5205adb-db84-4aab-95fe-5232a8468ed5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:02.789043 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:02.789778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 258/510] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Jun 7 08:22:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:02.790795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7884f79-c267-4116-a4cb-b68db3449e54 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:02.800047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4726d07-d44a-4306-a1bd-a14454112eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:02.800850 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4726d07-d44a-4306-a1bd-a14454112eac None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:02.801158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4726d07-d44a-4306-a1bd-a14454112eac None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:02.801440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4726d07-d44a-4306-a1bd-a14454112eac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:02.813257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4726d07-d44a-4306-a1bd-a14454112eac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:02.814723 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4726d07-d44a-4306-a1bd-a14454112eac None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f4726d07-d44a-4306-a1bd-a14454112eac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:02.833473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:02.834405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:02.835571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:02.835571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:02.835571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:02.835571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:02.835571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 253/511] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:02.836085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4726d07-d44a-4306-a1bd-a14454112eac None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:02.845989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:02.910768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7884f79-c267-4116-a4cb-b68db3449e54 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:02.946632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ed14b-c19b-4d84-831b-0728b23723be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:02.947314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ed14b-c19b-4d84-831b-0728b23723be None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:02.947604 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ed14b-c19b-4d84-831b-0728b23723be None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:02.947884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ed14b-c19b-4d84-831b-0728b23723be None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:02.948615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ed14b-c19b-4d84-831b-0728b23723be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:02.949205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ed14b-c19b-4d84-831b-0728b23723be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:02.950876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ed14b-c19b-4d84-831b-0728b23723be None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:02.953602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4726d07-d44a-4306-a1bd-a14454112eac None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:02.966312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 259/512] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:02.967173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ed14b-c19b-4d84-831b-0728b23723be None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:02.990807 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1a951-4d4a-45e5-a69f-0b8ef63fcc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:02.991645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1a951-4d4a-45e5-a69f-0b8ef63fcc44 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:02.991886 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1a951-4d4a-45e5-a69f-0b8ef63fcc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:02.992166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1a951-4d4a-45e5-a69f-0b8ef63fcc44 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:02.992897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1a951-4d4a-45e5-a69f-0b8ef63fcc44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:02.993237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1a951-4d4a-45e5-a69f-0b8ef63fcc44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:02.994712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1a951-4d4a-45e5-a69f-0b8ef63fcc44 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:03.007330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 254/513] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:03.008164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1a951-4d4a-45e5-a69f-0b8ef63fcc44 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:03.049588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:03.070578 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ed14b-c19b-4d84-831b-0728b23723be None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:03.079165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:03.108155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869c255-9f36-42f7-8991-5b876a8edbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:03.108827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869c255-9f36-42f7-8991-5b876a8edbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:03.109136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869c255-9f36-42f7-8991-5b876a8edbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:03.109422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869c255-9f36-42f7-8991-5b876a8edbd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:03.110179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869c255-9f36-42f7-8991-5b876a8edbd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:03.110486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869c255-9f36-42f7-8991-5b876a8edbd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:03.111880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869c255-9f36-42f7-8991-5b876a8edbd4 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:03.112723 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1a951-4d4a-45e5-a69f-0b8ef63fcc44 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:03.124098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 260/514] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:03.125021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869c255-9f36-42f7-8991-5b876a8edbd4 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:03.155001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c62696-b9d7-46ba-a7fe-ef3b1618f748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:03.155578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c62696-b9d7-46ba-a7fe-ef3b1618f748 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:03.155861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c62696-b9d7-46ba-a7fe-ef3b1618f748 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:03.156135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c62696-b9d7-46ba-a7fe-ef3b1618f748 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:03.232610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869c255-9f36-42f7-8991-5b876a8edbd4 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:03.239657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c62696-b9d7-46ba-a7fe-ef3b1618f748 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:03.241135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c62696-b9d7-46ba-a7fe-ef3b1618f748 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:03.267844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb63b7f-b585-46d4-a861-81dfcb65d11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:03.268510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb63b7f-b585-46d4-a861-81dfcb65d11f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:03.268792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb63b7f-b585-46d4-a861-81dfcb65d11f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:03.269067 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb63b7f-b585-46d4-a861-81dfcb65d11f None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:03.269830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb63b7f-b585-46d4-a861-81dfcb65d11f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:03.270134 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb63b7f-b585-46d4-a861-81dfcb65d11f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:03.271525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb63b7f-b585-46d4-a861-81dfcb65d11f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:03.283054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:03.284061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 261/515] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:03.284718 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb63b7f-b585-46d4-a861-81dfcb65d11f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:03.314864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:03.354599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 255/516] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:22:03.355735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c62696-b9d7-46ba-a7fe-ef3b1618f748 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:03.420462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb63b7f-b585-46d4-a861-81dfcb65d11f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:03.462176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8637d7-792b-4b6b-80f2-0b2f10e09a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:03.462176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8637d7-792b-4b6b-80f2-0b2f10e09a0b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:03.462176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8637d7-792b-4b6b-80f2-0b2f10e09a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:03.462176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8637d7-792b-4b6b-80f2-0b2f10e09a0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:03.470088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8637d7-792b-4b6b-80f2-0b2f10e09a0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:03.470755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c62696-b9d7-46ba-a7fe-ef3b1618f748 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:03.471571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8637d7-792b-4b6b-80f2-0b2f10e09a0b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-4a8637d7-792b-4b6b-80f2-0b2f10e09a0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:03.487726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:03.488862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:03.489823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:03.489823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:03.489823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:03.489823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:03.489823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:03.489823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:03.489823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:03.489823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:03.489823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 262/517] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:03.490306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8637d7-792b-4b6b-80f2-0b2f10e09a0b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:03.517961 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d11f50-1544-40d2-a746-6766e63bef29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:03.518768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d11f50-1544-40d2-a746-6766e63bef29 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:03.519054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d11f50-1544-40d2-a746-6766e63bef29 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:03.519333 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d11f50-1544-40d2-a746-6766e63bef29 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:03.526979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d11f50-1544-40d2-a746-6766e63bef29 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:03.528450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d11f50-1544-40d2-a746-6766e63bef29 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:03.530391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-a1d11f50-1544-40d2-a746-6766e63bef29 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:03.544744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:03.545728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:03.546609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:03.546609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:03.546609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:03.546609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:03.546609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:03.546609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:03.546609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:03.546609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:03.546609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 08:22:03.546609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:03.546609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 256/518] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:03.547182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d11f50-1544-40d2-a746-6766e63bef29 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:03.596555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:03.616076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8637d7-792b-4b6b-80f2-0b2f10e09a0b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:03.657904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b953b-9136-4fd0-b003-e4469f30f3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:03.658328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d11f50-1544-40d2-a746-6766e63bef29 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:03.658705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b953b-9136-4fd0-b003-e4469f30f3bc None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:03.658992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b953b-9136-4fd0-b003-e4469f30f3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:03.659265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b953b-9136-4fd0-b003-e4469f30f3bc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:03.667803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b953b-9136-4fd0-b003-e4469f30f3bc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:03.670106 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b953b-9136-4fd0-b003-e4469f30f3bc None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-e55b953b-9136-4fd0-b003-e4469f30f3bc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:03.685430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:03.686925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:03.688674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:03.688674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:03.688674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:03.688674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:03.688674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:03.688674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:03.688674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:03.688674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:03.688674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 08:22:03.688674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:03.688674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 263/519] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:03.688674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b953b-9136-4fd0-b003-e4469f30f3bc None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:03.705185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cbac2-04fc-4bd8-bcec-90a787a0c8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:03.705863 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cbac2-04fc-4bd8-bcec-90a787a0c8cc None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:03.706147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cbac2-04fc-4bd8-bcec-90a787a0c8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:03.706428 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cbac2-04fc-4bd8-bcec-90a787a0c8cc None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:03.707129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cbac2-04fc-4bd8-bcec-90a787a0c8cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:03.708562 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cbac2-04fc-4bd8-bcec-90a787a0c8cc None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:03.729041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 257/520] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:22:03 2022] POST /identity/v3/users => generated 247 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:22:03.729743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cbac2-04fc-4bd8-bcec-90a787a0c8cc None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:03.814362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b953b-9136-4fd0-b003-e4469f30f3bc None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:03.819047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:03.858844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96586f-ecd8-4f4f-9338-3bc12b152d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:03.859881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96586f-ecd8-4f4f-9338-3bc12b152d91 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:03.860424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96586f-ecd8-4f4f-9338-3bc12b152d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:03.860424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96586f-ecd8-4f4f-9338-3bc12b152d91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:03.874077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96586f-ecd8-4f4f-9338-3bc12b152d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:03.875720 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96586f-ecd8-4f4f-9338-3bc12b152d91 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:03.879445 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cbac2-04fc-4bd8-bcec-90a787a0c8cc None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-9f96586f-ecd8-4f4f-9338-3bc12b152d91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:03.900063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:03.900997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:03.901900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:03.901900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:03.901900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:03.901900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:03.901900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 264/521] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:03.903245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96586f-ecd8-4f4f-9338-3bc12b152d91 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:03.907354 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:03.937534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97da90-631c-4ddd-928e-c3ffe846d15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:03.938516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97da90-631c-4ddd-928e-c3ffe846d15e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:03.938805 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97da90-631c-4ddd-928e-c3ffe846d15e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:03.939087 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97da90-631c-4ddd-928e-c3ffe846d15e None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:03.939892 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97da90-631c-4ddd-928e-c3ffe846d15e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:03.940203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97da90-631c-4ddd-928e-c3ffe846d15e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:03.941694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97da90-631c-4ddd-928e-c3ffe846d15e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:03.963175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 258/522] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:03.963941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97da90-631c-4ddd-928e-c3ffe846d15e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:04.044412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96586f-ecd8-4f4f-9338-3bc12b152d91 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:04.086867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb22e3-26bc-4bde-80f7-d46fe9864a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:04.087551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb22e3-26bc-4bde-80f7-d46fe9864a32 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:04.087839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb22e3-26bc-4bde-80f7-d46fe9864a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:04.088112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb22e3-26bc-4bde-80f7-d46fe9864a32 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:04.088956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb22e3-26bc-4bde-80f7-d46fe9864a32 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:04.089309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb22e3-26bc-4bde-80f7-d46fe9864a32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:04.091305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb22e3-26bc-4bde-80f7-d46fe9864a32 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:04.093229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97da90-631c-4ddd-928e-c3ffe846d15e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:04.110920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 265/523] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Jun 7 08:22:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:04.111777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb22e3-26bc-4bde-80f7-d46fe9864a32 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:04.115768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:04.140587 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06f07b-4755-470c-a177-ee975e97f4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:04.141328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06f07b-4755-470c-a177-ee975e97f4ff None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:04.141648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06f07b-4755-470c-a177-ee975e97f4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:04.141888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06f07b-4755-470c-a177-ee975e97f4ff None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:04.142751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06f07b-4755-470c-a177-ee975e97f4ff None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:04.143062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06f07b-4755-470c-a177-ee975e97f4ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:04.144760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06f07b-4755-470c-a177-ee975e97f4ff None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:04.166181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 259/524] 10.222.0.39 () {56 vars in 1139 bytes} [Tue Jun 7 08:22:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:04.167194 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06f07b-4755-470c-a177-ee975e97f4ff None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:04.190030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:04.242313 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb22e3-26bc-4bde-80f7-d46fe9864a32 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:04.280169 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd6c95-4d3c-4d97-8548-c44482b56d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:04.280873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd6c95-4d3c-4d97-8548-c44482b56d92 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:04.281151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd6c95-4d3c-4d97-8548-c44482b56d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:04.281577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd6c95-4d3c-4d97-8548-c44482b56d92 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:04.282163 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd6c95-4d3c-4d97-8548-c44482b56d92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:04.282476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd6c95-4d3c-4d97-8548-c44482b56d92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:04.283880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd6c95-4d3c-4d97-8548-c44482b56d92 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:04.301846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 266/525] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:04.301846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd6c95-4d3c-4d97-8548-c44482b56d92 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:04.317964 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06f07b-4755-470c-a177-ee975e97f4ff None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:04.367558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc48d84-b3eb-4b2e-aeea-d9a305cb042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:04.368438 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc48d84-b3eb-4b2e-aeea-d9a305cb042e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:04.368735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc48d84-b3eb-4b2e-aeea-d9a305cb042e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:04.369009 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc48d84-b3eb-4b2e-aeea-d9a305cb042e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:04.370322 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc48d84-b3eb-4b2e-aeea-d9a305cb042e None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=3f2a2bf913d1412f996acf4401ba7673) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:04.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc48d84-b3eb-4b2e-aeea-d9a305cb042e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:04.372213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc48d84-b3eb-4b2e-aeea-d9a305cb042e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:04.379837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 260/526] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:04 2022] GET /identity/v3/role_assignments?user.id=3f2a2bf913d1412f996acf4401ba7673&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:04.381307 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc48d84-b3eb-4b2e-aeea-d9a305cb042e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:04.444174 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:04.454058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd6c95-4d3c-4d97-8548-c44482b56d92 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:04.477230 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:04.504233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f343fb-edb0-4d4b-b8ea-aae95dcdf380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:04.505012 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f343fb-edb0-4d4b-b8ea-aae95dcdf380 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:04.505409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f343fb-edb0-4d4b-b8ea-aae95dcdf380 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:04.505686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f343fb-edb0-4d4b-b8ea-aae95dcdf380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:04.541518 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc48d84-b3eb-4b2e-aeea-d9a305cb042e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:04.588188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee91ae0-23b7-4d08-a1b4-efbcfe0c63f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:04.588876 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee91ae0-23b7-4d08-a1b4-efbcfe0c63f0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:04.589328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee91ae0-23b7-4d08-a1b4-efbcfe0c63f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:04.589620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee91ae0-23b7-4d08-a1b4-efbcfe0c63f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:04.606238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f343fb-edb0-4d4b-b8ea-aae95dcdf380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:04.608246 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f343fb-edb0-4d4b-b8ea-aae95dcdf380 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:04.690556 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee91ae0-23b7-4d08-a1b4-efbcfe0c63f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:04.692341 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee91ae0-23b7-4d08-a1b4-efbcfe0c63f0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:04.729380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 267/527] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:22:04.730204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f343fb-edb0-4d4b-b8ea-aae95dcdf380 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:04.744919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:04.753232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:04.839569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 261/528] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:22:04.841003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee91ae0-23b7-4d08-a1b4-efbcfe0c63f0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:04.866564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f343fb-edb0-4d4b-b8ea-aae95dcdf380 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:04.917546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff02319-3e61-48a7-9f52-ed56020fb063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:04.920277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff02319-3e61-48a7-9f52-ed56020fb063 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:04.920836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff02319-3e61-48a7-9f52-ed56020fb063 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:04.921133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff02319-3e61-48a7-9f52-ed56020fb063 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:04.931389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff02319-3e61-48a7-9f52-ed56020fb063 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:04.932870 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff02319-3e61-48a7-9f52-ed56020fb063 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-7ff02319-3e61-48a7-9f52-ed56020fb063 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:04.955363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:04.956953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:04.958593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:04.958593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:04.958593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:04.958593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:04.958593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:04.958593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:04.958593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:04.958593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:04.965168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 268/529] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:04.965168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff02319-3e61-48a7-9f52-ed56020fb063 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:04.972315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee91ae0-23b7-4d08-a1b4-efbcfe0c63f0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:05.011765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb939d-0dec-4f2c-ae26-67ced4c95dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:05.013562 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb939d-0dec-4f2c-ae26-67ced4c95dca None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:05.013844 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb939d-0dec-4f2c-ae26-67ced4c95dca None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:05.014109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb939d-0dec-4f2c-ae26-67ced4c95dca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:05.017437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:05.029382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb939d-0dec-4f2c-ae26-67ced4c95dca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:05.030876 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb939d-0dec-4f2c-ae26-67ced4c95dca None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-a5fb939d-0dec-4f2c-ae26-67ced4c95dca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:05.050062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:05.050995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:05.052291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:05.052291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:05.052291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:05.052291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:05.056033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 262/530] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:05.057783 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb939d-0dec-4f2c-ae26-67ced4c95dca None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:05.066561 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:05.096424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff02319-3e61-48a7-9f52-ed56020fb063 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:05.134447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943c061-e9fc-4315-b85e-5cf22c4925b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:05.135657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943c061-e9fc-4315-b85e-5cf22c4925b9 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:05.136179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943c061-e9fc-4315-b85e-5cf22c4925b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:05.136651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943c061-e9fc-4315-b85e-5cf22c4925b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:05.149788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9943c061-e9fc-4315-b85e-5cf22c4925b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:05.151818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9943c061-e9fc-4315-b85e-5cf22c4925b9 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-9943c061-e9fc-4315-b85e-5cf22c4925b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:05.172259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:05.173214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:05.174049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:05.174049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:05.174049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:05.174049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:05.174267 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb939d-0dec-4f2c-ae26-67ced4c95dca None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:05.181013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 269/531] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:05.182333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943c061-e9fc-4315-b85e-5cf22c4925b9 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:05.213938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e748fa-76bd-4c90-a142-88c23b13799e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:05.214603 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e748fa-76bd-4c90-a142-88c23b13799e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:05.214943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e748fa-76bd-4c90-a142-88c23b13799e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:05.215293 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e748fa-76bd-4c90-a142-88c23b13799e None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:05.216123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e748fa-76bd-4c90-a142-88c23b13799e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:05.216434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e748fa-76bd-4c90-a142-88c23b13799e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:05.218037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e748fa-76bd-4c90-a142-88c23b13799e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:05.237042 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 263/532] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:05.237042 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e748fa-76bd-4c90-a142-88c23b13799e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:05.289597 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:05.308576 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943c061-e9fc-4315-b85e-5cf22c4925b9 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:05.339748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:05.349826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba4a2c-dbb5-4d77-9d0d-8c48ad35cc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:05.350848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba4a2c-dbb5-4d77-9d0d-8c48ad35cc5b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:05.350956 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e748fa-76bd-4c90-a142-88c23b13799e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:05.351596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba4a2c-dbb5-4d77-9d0d-8c48ad35cc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:05.352085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba4a2c-dbb5-4d77-9d0d-8c48ad35cc5b None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:05.352999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba4a2c-dbb5-4d77-9d0d-8c48ad35cc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:05.353529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba4a2c-dbb5-4d77-9d0d-8c48ad35cc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:05.355142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba4a2c-dbb5-4d77-9d0d-8c48ad35cc5b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:05.369914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 270/533] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:05.371234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba4a2c-dbb5-4d77-9d0d-8c48ad35cc5b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:05.394629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff566d4d-a632-4fb6-a885-51ac8b08bc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:05.395828 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff566d4d-a632-4fb6-a885-51ac8b08bc93 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:05.395828 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff566d4d-a632-4fb6-a885-51ac8b08bc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:05.396104 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff566d4d-a632-4fb6-a885-51ac8b08bc93 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:05.396792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff566d4d-a632-4fb6-a885-51ac8b08bc93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:05.397127 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff566d4d-a632-4fb6-a885-51ac8b08bc93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:05.398824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff566d4d-a632-4fb6-a885-51ac8b08bc93 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:05.417023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 264/534] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:05.418051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff566d4d-a632-4fb6-a885-51ac8b08bc93 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:05.484237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba4a2c-dbb5-4d77-9d0d-8c48ad35cc5b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:05.522577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04751b41-fc6f-4020-9206-086c5403521a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:05.523491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04751b41-fc6f-4020-9206-086c5403521a None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:05.523916 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04751b41-fc6f-4020-9206-086c5403521a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:05.524331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04751b41-fc6f-4020-9206-086c5403521a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:05.525370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04751b41-fc6f-4020-9206-086c5403521a None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=2b6943045d594d729fc7c4b15cbaa07e) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:05.525835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04751b41-fc6f-4020-9206-086c5403521a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:05.527546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04751b41-fc6f-4020-9206-086c5403521a None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:05.534815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 271/535] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:05 2022] GET /identity/v3/role_assignments?user.id=2b6943045d594d729fc7c4b15cbaa07e&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:05.542380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04751b41-fc6f-4020-9206-086c5403521a None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:05.549822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff566d4d-a632-4fb6-a885-51ac8b08bc93 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:05.573124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:05.599472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7f160-97af-49e9-ba31-815ecafc154a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:05.601553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7f160-97af-49e9-ba31-815ecafc154a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:05.601841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7f160-97af-49e9-ba31-815ecafc154a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:05.602123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7f160-97af-49e9-ba31-815ecafc154a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:05.621274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7f160-97af-49e9-ba31-815ecafc154a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:05.628552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7f160-97af-49e9-ba31-815ecafc154a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-12e7f160-97af-49e9-ba31-815ecafc154a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:05.677401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:05.678398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:05.679659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:05.679659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:05.679659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:05.679659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:05.679659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:05.679659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:05.679659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:05.679659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:05.679659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 265/536] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:05.688858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:05.695265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7f160-97af-49e9-ba31-815ecafc154a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:05.809042 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04751b41-fc6f-4020-9206-086c5403521a None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:05.859239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e6dd2-40af-4cb5-b944-0aca02a2d8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:05.860408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e6dd2-40af-4cb5-b944-0aca02a2d8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:05.860890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e6dd2-40af-4cb5-b944-0aca02a2d8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:05.861419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e6dd2-40af-4cb5-b944-0aca02a2d8e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:05.870607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e6dd2-40af-4cb5-b944-0aca02a2d8e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:05.872752 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e6dd2-40af-4cb5-b944-0aca02a2d8e7 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-c61e6dd2-40af-4cb5-b944-0aca02a2d8e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:05.890298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:05.891267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:05.892630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:05.892630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:05.892630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:05.892630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:05.892630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:05.892630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:05.892630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:05.892630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:05.895067 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 272/537] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:05.904650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e6dd2-40af-4cb5-b944-0aca02a2d8e7 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:05.908541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7f160-97af-49e9-ba31-815ecafc154a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:06.027694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e6dd2-40af-4cb5-b944-0aca02a2d8e7 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:06.049468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:06.049900 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3bdb0-b326-4119-b8d2-695eb08a9fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:06.050611 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3bdb0-b326-4119-b8d2-695eb08a9fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:06.050896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3bdb0-b326-4119-b8d2-695eb08a9fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:06.051168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3bdb0-b326-4119-b8d2-695eb08a9fd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:06.052057 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3bdb0-b326-4119-b8d2-695eb08a9fd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:06.052305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3bdb0-b326-4119-b8d2-695eb08a9fd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:06.053882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3bdb0-b326-4119-b8d2-695eb08a9fd5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:06.077539 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e6150-a65f-4b99-9f8e-488bf4b69529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:06.079147 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e6150-a65f-4b99-9f8e-488bf4b69529 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:06.079626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e6150-a65f-4b99-9f8e-488bf4b69529 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:06.080094 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e6150-a65f-4b99-9f8e-488bf4b69529 None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:06.081146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e6150-a65f-4b99-9f8e-488bf4b69529 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:06.081680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e6150-a65f-4b99-9f8e-488bf4b69529 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:06.083575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e6150-a65f-4b99-9f8e-488bf4b69529 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:06.089989 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 266/538] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:06.096924 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3bdb0-b326-4119-b8d2-695eb08a9fd5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:06.125251 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 273/539] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:06.127159 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e6150-a65f-4b99-9f8e-488bf4b69529 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:06.173477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:06.311882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3bdb0-b326-4119-b8d2-695eb08a9fd5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:06.341809 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e6150-a65f-4b99-9f8e-488bf4b69529 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:06.363897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628807e-12cc-4546-b18e-d0c429d85670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:06.364573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628807e-12cc-4546-b18e-d0c429d85670 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:06.364866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628807e-12cc-4546-b18e-d0c429d85670 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:06.365149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628807e-12cc-4546-b18e-d0c429d85670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:06.365994 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628807e-12cc-4546-b18e-d0c429d85670 None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=d1e4531ac696445ba5bfa9870984befd) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:06.366305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628807e-12cc-4546-b18e-d0c429d85670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:06.368162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628807e-12cc-4546-b18e-d0c429d85670 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:06.377472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 267/540] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:06 2022] GET /identity/v3/role_assignments?user.id=d1e4531ac696445ba5bfa9870984befd&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:06.378916 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33490c8-cf3f-4210-8ba5-c5e847722f86 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:06.379206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33490c8-cf3f-4210-8ba5-c5e847722f86 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:06.379475 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33490c8-cf3f-4210-8ba5-c5e847722f86 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:06.380860 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 268/541] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:06.387123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c351df-97b8-41b9-a697-60596df233dc None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:06.387888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c351df-97b8-41b9-a697-60596df233dc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:06.388352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c351df-97b8-41b9-a697-60596df233dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:06.407165 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-60c351df-97b8-41b9-a697-60596df233dc None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:06.408633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-60c351df-97b8-41b9-a697-60596df233dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:06.428172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb120d-9411-4272-9f92-594113e54145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:06.429493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb120d-9411-4272-9f92-594113e54145 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:06.430017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb120d-9411-4272-9f92-594113e54145 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:06.430594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb120d-9411-4272-9f92-594113e54145 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:06.432634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb120d-9411-4272-9f92-594113e54145 None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=7ddf89deb28749eda2d26ad66bc282f8) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:06.433394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb120d-9411-4272-9f92-594113e54145 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:06.442689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb120d-9411-4272-9f92-594113e54145 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:06.461586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 274/542] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:06 2022] GET /identity/v3/role_assignments?user.id=7ddf89deb28749eda2d26ad66bc282f8&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:06.525286 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 269/543] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:22:06.567677 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb120d-9411-4272-9f92-594113e54145 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:06.650912 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:06.813908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb120d-9411-4272-9f92-594113e54145 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:06.876750 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66db014-707d-434a-8e31-b5a7feb8900c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:06.877687 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66db014-707d-434a-8e31-b5a7feb8900c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:06.877968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66db014-707d-434a-8e31-b5a7feb8900c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:06.878252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66db014-707d-434a-8e31-b5a7feb8900c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:06.905209 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66db014-707d-434a-8e31-b5a7feb8900c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-d66db014-707d-434a-8e31-b5a7feb8900c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:06.914365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:06.916517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:06.918053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:06.918053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:06.918053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:06.918053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:06.918053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:06.918053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:06.918053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:06.918053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:06.918053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:06.918053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:06.918053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:06.918053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:06.918053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 275/544] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:06.930857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c351df-97b8-41b9-a697-60596df233dc None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:06.998688 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:07.086772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fa1e1-677f-46c8-ad96-146f972676f8 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:07.087292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fa1e1-677f-46c8-ad96-146f972676f8 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:07.087398 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fa1e1-677f-46c8-ad96-146f972676f8 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:07.088970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 276/545] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:07.103745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db520ead-7c37-4c7c-b592-b7047899c35d None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:07.104199 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db520ead-7c37-4c7c-b592-b7047899c35d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:07.104664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db520ead-7c37-4c7c-b592-b7047899c35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:07.124605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c351df-97b8-41b9-a697-60596df233dc None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:07.150524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-db520ead-7c37-4c7c-b592-b7047899c35d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:07.153098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-db520ead-7c37-4c7c-b592-b7047899c35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:07.205713 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc54db-415f-4ab5-91b3-278c3be4f69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:07.206319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc54db-415f-4ab5-91b3-278c3be4f69d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:07.206605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc54db-415f-4ab5-91b3-278c3be4f69d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:07.206888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc54db-415f-4ab5-91b3-278c3be4f69d None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:07.207645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc54db-415f-4ab5-91b3-278c3be4f69d None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:07.207948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc54db-415f-4ab5-91b3-278c3be4f69d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:07.217152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc54db-415f-4ab5-91b3-278c3be4f69d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:07.219444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 270/546] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:07.228217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc54db-415f-4ab5-91b3-278c3be4f69d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:07.296297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 277/547] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:07.348278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db520ead-7c37-4c7c-b592-b7047899c35d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:07.405440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:07.429608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc54db-415f-4ab5-91b3-278c3be4f69d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:07.436511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:07.482880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df36fa3-1b1c-4314-a21d-18e886553a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:07.483948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df36fa3-1b1c-4314-a21d-18e886553a55 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:07.484382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df36fa3-1b1c-4314-a21d-18e886553a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:07.484822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df36fa3-1b1c-4314-a21d-18e886553a55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:07.501498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df36fa3-1b1c-4314-a21d-18e886553a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:07.503201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df36fa3-1b1c-4314-a21d-18e886553a55 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-5df36fa3-1b1c-4314-a21d-18e886553a55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:07.563059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:07.564541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:07.565963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:07.565963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:07.565963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:07.565963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:07.575244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 271/548] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:07.582816 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df36fa3-1b1c-4314-a21d-18e886553a55 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:07.678181 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db520ead-7c37-4c7c-b592-b7047899c35d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:07.822361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99759da-b8a4-4722-ba4e-e2004d10ed83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:07.823704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99759da-b8a4-4722-ba4e-e2004d10ed83 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:07.824226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99759da-b8a4-4722-ba4e-e2004d10ed83 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:07.825333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99759da-b8a4-4722-ba4e-e2004d10ed83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:07.858180 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99759da-b8a4-4722-ba4e-e2004d10ed83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-c99759da-b8a4-4722-ba4e-e2004d10ed83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:07.874080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:07.876669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:07.879029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:07.879029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:07.879029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:07.879029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:07.879029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:07.879029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:07.879029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:07.879029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:07.879029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:07.879029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:07.879029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:07.879029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:07.879029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 278/549] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 528 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:07.881375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99759da-b8a4-4722-ba4e-e2004d10ed83 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:07.906509 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df36fa3-1b1c-4314-a21d-18e886553a55 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:07.948555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7360711-7997-4504-b322-49aeb1d69e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:07.950844 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7360711-7997-4504-b322-49aeb1d69e0f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:07.951533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7360711-7997-4504-b322-49aeb1d69e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:07.952169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7360711-7997-4504-b322-49aeb1d69e0f None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:07.954287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7360711-7997-4504-b322-49aeb1d69e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:07.955432 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7360711-7997-4504-b322-49aeb1d69e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:07.958708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7360711-7997-4504-b322-49aeb1d69e0f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:07.964692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:07.987952 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 272/550] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:07.994262 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7360711-7997-4504-b322-49aeb1d69e0f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:08.096746 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:08.123980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99759da-b8a4-4722-ba4e-e2004d10ed83 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:08.176355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ac97e-3669-494e-b416-501e403700ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:08.177056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ac97e-3669-494e-b416-501e403700ca None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:08.177364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ac97e-3669-494e-b416-501e403700ca None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:08.177757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ac97e-3669-494e-b416-501e403700ca None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:08.178551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ac97e-3669-494e-b416-501e403700ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:08.179175 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ac97e-3669-494e-b416-501e403700ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:08.180766 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ac97e-3669-494e-b416-501e403700ca None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:08.192787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 279/551] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:08.200288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ac97e-3669-494e-b416-501e403700ca None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:08.263788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7360711-7997-4504-b322-49aeb1d69e0f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:08.327504 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5a274-d6b7-448b-8635-645339bba4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:08.328584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5a274-d6b7-448b-8635-645339bba4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:08.328870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5a274-d6b7-448b-8635-645339bba4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:08.329182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5a274-d6b7-448b-8635-645339bba4e2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:08.340998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5a274-d6b7-448b-8635-645339bba4e2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:08.352759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5a274-d6b7-448b-8635-645339bba4e2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:08.366193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ac97e-3669-494e-b416-501e403700ca None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-06d5a274-d6b7-448b-8635-645339bba4e2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:08.380721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:08.381870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:08.382790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:08.382790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:08.382790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:08.382790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:08.382790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:08.382790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:08.382790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:08.382790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:08.382790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:22:08.382790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:08.389229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 273/552] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:08.390389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5a274-d6b7-448b-8635-645339bba4e2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:08.434631 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701ed7b-c768-4e7e-a701-5e2d7056c01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:08.435968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701ed7b-c768-4e7e-a701-5e2d7056c01c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:08.436451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701ed7b-c768-4e7e-a701-5e2d7056c01c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:08.436928 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701ed7b-c768-4e7e-a701-5e2d7056c01c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:08.457966 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701ed7b-c768-4e7e-a701-5e2d7056c01c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:08.460697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701ed7b-c768-4e7e-a701-5e2d7056c01c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:08.477399 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-b701ed7b-c768-4e7e-a701-5e2d7056c01c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:08.500275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:08.501269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:08.502210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:08.502210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:08.502210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:08.502210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:08.502210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 280/553] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:08.514390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701ed7b-c768-4e7e-a701-5e2d7056c01c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:08.541619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:08.630512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5a274-d6b7-448b-8635-645339bba4e2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:08.702107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52460f3-554f-4971-8184-5df54d2bbc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:08.703715 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52460f3-554f-4971-8184-5df54d2bbc42 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:08.704114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52460f3-554f-4971-8184-5df54d2bbc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:08.704399 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52460f3-554f-4971-8184-5df54d2bbc42 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:08.705343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52460f3-554f-4971-8184-5df54d2bbc42 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:08.705658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52460f3-554f-4971-8184-5df54d2bbc42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:08.706644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701ed7b-c768-4e7e-a701-5e2d7056c01c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:08.707213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52460f3-554f-4971-8184-5df54d2bbc42 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:08.730391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 274/554] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Jun 7 08:22:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:08.737116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52460f3-554f-4971-8184-5df54d2bbc42 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:08.778582 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3d5f9-6972-400e-a3c1-1e03b89717b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:08.779751 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3d5f9-6972-400e-a3c1-1e03b89717b7 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:08.780248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3d5f9-6972-400e-a3c1-1e03b89717b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:08.780785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3d5f9-6972-400e-a3c1-1e03b89717b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:08.786788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3d5f9-6972-400e-a3c1-1e03b89717b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:08.787578 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3d5f9-6972-400e-a3c1-1e03b89717b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:08.789586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3d5f9-6972-400e-a3c1-1e03b89717b7 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:08.814992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 281/555] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:08.821858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3d5f9-6972-400e-a3c1-1e03b89717b7 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:08.876717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52460f3-554f-4971-8184-5df54d2bbc42 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:08.899082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:08.917923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517ac01-1c03-4793-85db-fb17505e1d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:08.919146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517ac01-1c03-4793-85db-fb17505e1d55 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:08.919440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517ac01-1c03-4793-85db-fb17505e1d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:08.919824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517ac01-1c03-4793-85db-fb17505e1d55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:08.953026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:09.010772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517ac01-1c03-4793-85db-fb17505e1d55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:09.013121 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517ac01-1c03-4793-85db-fb17505e1d55 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:09.017629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3d5f9-6972-400e-a3c1-1e03b89717b7 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:09.069716 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38560fc7-a2a5-4a67-9d62-88afeb87d33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:09.070771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38560fc7-a2a5-4a67-9d62-88afeb87d33e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:09.071100 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38560fc7-a2a5-4a67-9d62-88afeb87d33e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:09.071378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38560fc7-a2a5-4a67-9d62-88afeb87d33e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:09.083882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38560fc7-a2a5-4a67-9d62-88afeb87d33e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:09.086353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38560fc7-a2a5-4a67-9d62-88afeb87d33e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-38560fc7-a2a5-4a67-9d62-88afeb87d33e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:09.108085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:09.109979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:09.110881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:09.110881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:09.110881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:09.110881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:09.110881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:09.110881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:09.110881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:09.110881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:09.110881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:22:09.110881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:09.114184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 282/556] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jun 7 08:22:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:09.115769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710f30a-4ecb-4c61-a0f7-a326b91d660c None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:09.116204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710f30a-4ecb-4c61-a0f7-a326b91d660c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:09.116604 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710f30a-4ecb-4c61-a0f7-a326b91d660c None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:09.123319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 283/557] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:09.123319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710f30a-4ecb-4c61-a0f7-a326b91d660c None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:09.165416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 275/558] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:22:09.167382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f01963-ceeb-4231-bc08-1d042f7ff5da None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:09.167679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f01963-ceeb-4231-bc08-1d042f7ff5da None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:09.167959 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f01963-ceeb-4231-bc08-1d042f7ff5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:09.186334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a5f01963-ceeb-4231-bc08-1d042f7ff5da None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:09.187559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a5f01963-ceeb-4231-bc08-1d042f7ff5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:09.198539 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:09.252200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710f30a-4ecb-4c61-a0f7-a326b91d660c None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:09.272698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 276/559] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:09.272698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f01963-ceeb-4231-bc08-1d042f7ff5da None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:09.290438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:09.319842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf90a0-c1ba-457e-b990-c31d4f163b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:09.320963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf90a0-c1ba-457e-b990-c31d4f163b41 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:09.321527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf90a0-c1ba-457e-b990-c31d4f163b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:09.322088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf90a0-c1ba-457e-b990-c31d4f163b41 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:09.323133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf90a0-c1ba-457e-b990-c31d4f163b41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:09.323742 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf90a0-c1ba-457e-b990-c31d4f163b41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:09.326679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf90a0-c1ba-457e-b990-c31d4f163b41 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:09.348461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 284/560] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jun 7 08:22:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:09.350025 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf90a0-c1ba-457e-b990-c31d4f163b41 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:09.381028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f01963-ceeb-4231-bc08-1d042f7ff5da None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:09.418944 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d3d13-5902-4da7-acac-aab04f0bb754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:09.419651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d3d13-5902-4da7-acac-aab04f0bb754 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:09.419941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d3d13-5902-4da7-acac-aab04f0bb754 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:09.420220 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d3d13-5902-4da7-acac-aab04f0bb754 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:09.430830 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d3d13-5902-4da7-acac-aab04f0bb754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:09.432326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d3d13-5902-4da7-acac-aab04f0bb754 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:09.435945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-b48d3d13-5902-4da7-acac-aab04f0bb754 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:09.454158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:09.455301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:09.456212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:09.456212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:09.456212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:09.456212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:09.456212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 277/561] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:09.456650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a477331-51bd-4e17-973c-36c98b1eca3d None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:09.456938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a477331-51bd-4e17-973c-36c98b1eca3d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:09.457296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a477331-51bd-4e17-973c-36c98b1eca3d None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:09.458756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 278/562] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:09.461827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a477331-51bd-4e17-973c-36c98b1eca3d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:09.466086 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf90a0-c1ba-457e-b990-c31d4f163b41 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:09.506875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3b2a1-ec5e-4463-9fa3-48704c6be2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:09.507841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3b2a1-ec5e-4463-9fa3-48704c6be2ea None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:09.508043 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3b2a1-ec5e-4463-9fa3-48704c6be2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:09.508349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3b2a1-ec5e-4463-9fa3-48704c6be2ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:09.524066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3b2a1-ec5e-4463-9fa3-48704c6be2ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-e5c3b2a1-ec5e-4463-9fa3-48704c6be2ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:09.532303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:09.533505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:09.534504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:09.534504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:09.534504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:09.534504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:09.534504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:09.534504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:09.534504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:09.534504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:09.534504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:09.534504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:09.534504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:09.534504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:09.534504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 285/563] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:09.536970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af9708-dd4b-468c-943a-4156f86c31b5 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:09.537560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af9708-dd4b-468c-943a-4156f86c31b5 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:09.537560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af9708-dd4b-468c-943a-4156f86c31b5 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:09.539347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 286/564] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:09.540240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af9708-dd4b-468c-943a-4156f86c31b5 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:09.587523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a477331-51bd-4e17-973c-36c98b1eca3d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:09.588109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:09.625750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469620f7-0813-486d-93d9-3e8adf6f2cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:09.626551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469620f7-0813-486d-93d9-3e8adf6f2cca None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:09.626828 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469620f7-0813-486d-93d9-3e8adf6f2cca None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:09.627099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469620f7-0813-486d-93d9-3e8adf6f2cca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:09.640150 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469620f7-0813-486d-93d9-3e8adf6f2cca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:09.641673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469620f7-0813-486d-93d9-3e8adf6f2cca None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:09.658293 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af9708-dd4b-468c-943a-4156f86c31b5 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-469620f7-0813-486d-93d9-3e8adf6f2cca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:09.660831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:09.661911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:09.662950 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:09.662950 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:09.662950 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:09.662950 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:09.662950 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 279/565] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:09.663551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cbf70-2d96-4ab2-99b4-1220342895ca None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:09.663843 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cbf70-2d96-4ab2-99b4-1220342895ca None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:09.664116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cbf70-2d96-4ab2-99b4-1220342895ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:09.682254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-658cbf70-2d96-4ab2-99b4-1220342895ca None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:09.683387 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-658cbf70-2d96-4ab2-99b4-1220342895ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:09.698714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3ea5b-9a10-43f1-b690-b671242373f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:09.699403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3ea5b-9a10-43f1-b690-b671242373f0 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:09.699685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3ea5b-9a10-43f1-b690-b671242373f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:09.700123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3ea5b-9a10-43f1-b690-b671242373f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:09.701261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3ea5b-9a10-43f1-b690-b671242373f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:09.701572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3ea5b-9a10-43f1-b690-b671242373f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:09.703004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3ea5b-9a10-43f1-b690-b671242373f0 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:09.717004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 287/566] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:09.717920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3ea5b-9a10-43f1-b690-b671242373f0 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:09.737962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:09.764855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 280/567] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:09.766263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1efeb7-6850-4d9c-b984-b1c804364be3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:09.766559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1efeb7-6850-4d9c-b984-b1c804364be3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:09.766832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1efeb7-6850-4d9c-b984-b1c804364be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:09.794339 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0c1efeb7-6850-4d9c-b984-b1c804364be3 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:09.795823 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0c1efeb7-6850-4d9c-b984-b1c804364be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:09.826513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:09.831807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3ea5b-9a10-43f1-b690-b671242373f0 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:09.879527 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 281/568] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:09.880133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a91868-f347-482c-9d77-dbdc2e5dac5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:09.880406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1efeb7-6850-4d9c-b984-b1c804364be3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:09.881177 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a91868-f347-482c-9d77-dbdc2e5dac5b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:09.881513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a91868-f347-482c-9d77-dbdc2e5dac5b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:09.881739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a91868-f347-482c-9d77-dbdc2e5dac5b None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:09.882525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a91868-f347-482c-9d77-dbdc2e5dac5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:09.882835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a91868-f347-482c-9d77-dbdc2e5dac5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:09.887025 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a91868-f347-482c-9d77-dbdc2e5dac5b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:09.899695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 288/569] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:09.901228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a91868-f347-482c-9d77-dbdc2e5dac5b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:10.013629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1efeb7-6850-4d9c-b984-b1c804364be3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:10.015505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a91868-f347-482c-9d77-dbdc2e5dac5b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:10.023294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:10.051541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1f371-52ec-43cb-9980-56acc1f8f7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:10.052204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1f371-52ec-43cb-9980-56acc1f8f7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:10.052490 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1f371-52ec-43cb-9980-56acc1f8f7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:10.052758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1f371-52ec-43cb-9980-56acc1f8f7f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:10.053516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1f371-52ec-43cb-9980-56acc1f8f7f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:10.053688 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d802a08-a871-4e76-ac41-7c4df0f9b798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:10.053818 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1f371-52ec-43cb-9980-56acc1f8f7f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:10.054489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d802a08-a871-4e76-ac41-7c4df0f9b798 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:10.054774 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d802a08-a871-4e76-ac41-7c4df0f9b798 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:10.055053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d802a08-a871-4e76-ac41-7c4df0f9b798 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:10.055196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1f371-52ec-43cb-9980-56acc1f8f7f5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:10.062838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d802a08-a871-4e76-ac41-7c4df0f9b798 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:10.064307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d802a08-a871-4e76-ac41-7c4df0f9b798 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:10.067595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 282/570] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:10.068450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1f371-52ec-43cb-9980-56acc1f8f7f5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-8d802a08-a871-4e76-ac41-7c4df0f9b798 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:10.080264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:10.081497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:10.083228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:10.083228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:10.083228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:10.083228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:10.083228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:10.083228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:10.083228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:10.083228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:10.083228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 289/571] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:10.083228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d802a08-a871-4e76-ac41-7c4df0f9b798 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:10.107137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:10.179565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1f371-52ec-43cb-9980-56acc1f8f7f5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:10.194768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d802a08-a871-4e76-ac41-7c4df0f9b798 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:10.229439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd6952-79e5-417c-8ec9-8b563e96a75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:10.231674 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd6952-79e5-417c-8ec9-8b563e96a75d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:10.231674 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd6952-79e5-417c-8ec9-8b563e96a75d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:10.231674 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd6952-79e5-417c-8ec9-8b563e96a75d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:10.242620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd6952-79e5-417c-8ec9-8b563e96a75d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:10.250556 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633d3e4-eb4f-49cb-a042-a9480dd7a454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f6bd6952-79e5-417c-8ec9-8b563e96a75d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:10.250842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:10.251969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:10.257690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:10.257690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:10.257690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:10.257690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:10.257690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:10.257690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:10.257690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:10.257690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:10.257690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:10.257690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:10.257690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:10.257690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:10.258394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633d3e4-eb4f-49cb-a042-a9480dd7a454 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:10.258394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633d3e4-eb4f-49cb-a042-a9480dd7a454 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:10.258394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633d3e4-eb4f-49cb-a042-a9480dd7a454 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:10.258576 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 283/572] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:10.258576 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd6952-79e5-417c-8ec9-8b563e96a75d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:10.268783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633d3e4-eb4f-49cb-a042-a9480dd7a454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:10.270540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633d3e4-eb4f-49cb-a042-a9480dd7a454 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:10.290831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-f633d3e4-eb4f-49cb-a042-a9480dd7a454 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:10.291309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:10.292192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:10.293173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:10.293173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:10.293173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:10.293173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:10.297950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 290/573] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:10.298799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633d3e4-eb4f-49cb-a042-a9480dd7a454 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:10.368540 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd6952-79e5-417c-8ec9-8b563e96a75d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:10.373672 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:10.404222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727aea0-2453-4521-acc9-09a15e193c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:10.405031 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727aea0-2453-4521-acc9-09a15e193c51 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:10.405228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633d3e4-eb4f-49cb-a042-a9480dd7a454 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:10.405333 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727aea0-2453-4521-acc9-09a15e193c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:10.405608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727aea0-2453-4521-acc9-09a15e193c51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:10.416173 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727aea0-2453-4521-acc9-09a15e193c51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-0727aea0-2453-4521-acc9-09a15e193c51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:10.424361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:10.425465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:10.426309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:10.426309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:10.426309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:10.426309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:10.426309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:10.426309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:10.426309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:10.426309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:10.426309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:10.426309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:10.426309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:10.426309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:10.426309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 284/574] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:10.426984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727aea0-2453-4521-acc9-09a15e193c51 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:10.443124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0dd1e-94bd-4b40-ba86-52954fb121ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:10.443924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0dd1e-94bd-4b40-ba86-52954fb121ea None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:10.444210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0dd1e-94bd-4b40-ba86-52954fb121ea None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:10.444488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0dd1e-94bd-4b40-ba86-52954fb121ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:10.452194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0dd1e-94bd-4b40-ba86-52954fb121ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:10.453948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0dd1e-94bd-4b40-ba86-52954fb121ea None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-7ed0dd1e-94bd-4b40-ba86-52954fb121ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:10.469670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:10.470668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:10.471595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:10.471595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:10.471595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:10.471595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:10.471595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:10.471595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:10.471595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:10.471595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:10.471595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 291/575] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:10.472091 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0dd1e-94bd-4b40-ba86-52954fb121ea None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:10.524326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:10.539628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727aea0-2453-4521-acc9-09a15e193c51 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:10.581971 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c0a51-5fe0-4e89-9fec-ec0a673dd08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:10.582642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c0a51-5fe0-4e89-9fec-ec0a673dd08d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:10.582957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c0a51-5fe0-4e89-9fec-ec0a673dd08d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:10.583192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c0a51-5fe0-4e89-9fec-ec0a673dd08d None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:10.583986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c0a51-5fe0-4e89-9fec-ec0a673dd08d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:10.584290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c0a51-5fe0-4e89-9fec-ec0a673dd08d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:10.584518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0dd1e-94bd-4b40-ba86-52954fb121ea None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:10.585773 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c0a51-5fe0-4e89-9fec-ec0a673dd08d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:10.599997 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 285/576] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:10.600839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c0a51-5fe0-4e89-9fec-ec0a673dd08d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:10.620080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:10.627764 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c6cd0-f9c7-4f21-b896-b4552ad28244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:10.628556 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c6cd0-f9c7-4f21-b896-b4552ad28244 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:10.628885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c6cd0-f9c7-4f21-b896-b4552ad28244 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:10.629234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c6cd0-f9c7-4f21-b896-b4552ad28244 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:10.629994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c6cd0-f9c7-4f21-b896-b4552ad28244 None admin] RBAC: Adding query filter params (name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:10.630305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c6cd0-f9c7-4f21-b896-b4552ad28244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:10.631706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c6cd0-f9c7-4f21-b896-b4552ad28244 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:10.641683 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 292/577] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:10.642510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c6cd0-f9c7-4f21-b896-b4552ad28244 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:10.710629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c0a51-5fe0-4e89-9fec-ec0a673dd08d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:10.745543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bf5fc-4814-41a7-b054-5601cdf72824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:10.746007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c6cd0-f9c7-4f21-b896-b4552ad28244 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:10.746813 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bf5fc-4814-41a7-b054-5601cdf72824 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:10.747101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bf5fc-4814-41a7-b054-5601cdf72824 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:10.747377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bf5fc-4814-41a7-b054-5601cdf72824 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:10.748102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bf5fc-4814-41a7-b054-5601cdf72824 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:10.748444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bf5fc-4814-41a7-b054-5601cdf72824 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:10.749862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bf5fc-4814-41a7-b054-5601cdf72824 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:10.761304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 286/578] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:10.762205 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bf5fc-4814-41a7-b054-5601cdf72824 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:10.773718 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:10.788432 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c9a6c-459a-40d8-89ab-5370efb6b170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:10.789021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c9a6c-459a-40d8-89ab-5370efb6b170 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:10.789328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c9a6c-459a-40d8-89ab-5370efb6b170 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:10.789604 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c9a6c-459a-40d8-89ab-5370efb6b170 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:10.790341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c9a6c-459a-40d8-89ab-5370efb6b170 None admin] RBAC: Adding query filter params (name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:10.790646 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c9a6c-459a-40d8-89ab-5370efb6b170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:10.792006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c9a6c-459a-40d8-89ab-5370efb6b170 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:10.801123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 293/579] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:10.801965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c9a6c-459a-40d8-89ab-5370efb6b170 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:10.869660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:10.883377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bf5fc-4814-41a7-b054-5601cdf72824 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:10.937460 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784cdab-4508-4016-9bde-b1d01320dc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:10.938323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c9a6c-459a-40d8-89ab-5370efb6b170 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:10.938502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784cdab-4508-4016-9bde-b1d01320dc2f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:10.938952 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784cdab-4508-4016-9bde-b1d01320dc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:10.939410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784cdab-4508-4016-9bde-b1d01320dc2f None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:10.940771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784cdab-4508-4016-9bde-b1d01320dc2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:10.941323 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784cdab-4508-4016-9bde-b1d01320dc2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:10.943411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784cdab-4508-4016-9bde-b1d01320dc2f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:10.963180 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 287/580] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:10.964055 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784cdab-4508-4016-9bde-b1d01320dc2f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:10.985748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5d203-a791-441b-abd2-ae0ac0b56ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:10.986416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5d203-a791-441b-abd2-ae0ac0b56ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:10.986704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5d203-a791-441b-abd2-ae0ac0b56ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:10.986985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5d203-a791-441b-abd2-ae0ac0b56ee9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:10.987827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5d203-a791-441b-abd2-ae0ac0b56ee9 None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=586ddffa77a54521b42b2aa50f16cb92) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:10.988143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5d203-a791-441b-abd2-ae0ac0b56ee9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:10.989630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5d203-a791-441b-abd2-ae0ac0b56ee9 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:10.996848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 294/581] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:10 2022] GET /identity/v3/role_assignments?user.id=586ddffa77a54521b42b2aa50f16cb92&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:10.998132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5d203-a791-441b-abd2-ae0ac0b56ee9 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:11.055834 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:11.107955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784cdab-4508-4016-9bde-b1d01320dc2f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:11.138762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5d203-a791-441b-abd2-ae0ac0b56ee9 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:11.151376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071922bc-1805-4dd9-9dae-ede1a13010b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:11.152404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071922bc-1805-4dd9-9dae-ede1a13010b2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:11.153367 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071922bc-1805-4dd9-9dae-ede1a13010b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:11.153712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071922bc-1805-4dd9-9dae-ede1a13010b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:11.165622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:11.166478 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071922bc-1805-4dd9-9dae-ede1a13010b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:11.168170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071922bc-1805-4dd9-9dae-ede1a13010b2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:11.184808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ca586-0277-40fa-b412-7f4eeeb96ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:11.186033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ca586-0277-40fa-b412-7f4eeeb96ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:11.186479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ca586-0277-40fa-b412-7f4eeeb96ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:11.186943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ca586-0277-40fa-b412-7f4eeeb96ca9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-071922bc-1805-4dd9-9dae-ede1a13010b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:11.190049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:11.191161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:11.192096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:11.192096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:11.192096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:11.192096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:11.192096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 288/582] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:11.192504 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071922bc-1805-4dd9-9dae-ede1a13010b2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:11.204913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ca586-0277-40fa-b412-7f4eeeb96ca9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:11.207567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ca586-0277-40fa-b412-7f4eeeb96ca9 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-e22ca586-0277-40fa-b412-7f4eeeb96ca9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:11.225904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:11.227072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:11.234066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:11.234066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:11.234066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:11.234066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:11.234066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:11.234066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:11.234066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:11.234066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:11.234066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:22:11.234066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:11.234066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 295/583] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:11.234066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ca586-0277-40fa-b412-7f4eeeb96ca9 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:11.327668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071922bc-1805-4dd9-9dae-ede1a13010b2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:11.358197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:11.378290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0f373-2468-4b7e-a3b5-4b5f682ffa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:11.379200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0f373-2468-4b7e-a3b5-4b5f682ffa01 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:11.379563 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0f373-2468-4b7e-a3b5-4b5f682ffa01 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:11.379912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0f373-2468-4b7e-a3b5-4b5f682ffa01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:11.382889 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ca586-0277-40fa-b412-7f4eeeb96ca9 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:11.393068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0f373-2468-4b7e-a3b5-4b5f682ffa01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:11.394675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0f373-2468-4b7e-a3b5-4b5f682ffa01 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-75d0f373-2468-4b7e-a3b5-4b5f682ffa01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:11.415037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:11.416103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:11.417175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:11.417175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:11.417175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:11.417175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:11.421509 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 289/584] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:11.422621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0f373-2468-4b7e-a3b5-4b5f682ffa01 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:11.449784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62a208-38cb-48d6-bbe0-6049b432ae97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:11.452087 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62a208-38cb-48d6-bbe0-6049b432ae97 None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:11.452319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62a208-38cb-48d6-bbe0-6049b432ae97 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:11.452870 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62a208-38cb-48d6-bbe0-6049b432ae97 None admin] PATH_INFO: `/v3/projects/07b861f95bf3498d990785ba72474d82/users/3f2a2bf913d1412f996acf4401ba7673/roles/9f07c51b3716428b89a49a756664305a` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:11.484345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62a208-38cb-48d6-bbe0-6049b432ae97 None admin] RBAC: Authorizing `identity:create_grant(project_id=07b861f95bf3498d990785ba72474d82, user_id=3f2a2bf913d1412f996acf4401ba7673, role_id=9f07c51b3716428b89a49a756664305a)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:11.485865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62a208-38cb-48d6-bbe0-6049b432ae97 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:11.506712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 296/585] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:22:11 2022] PUT /identity/v3/projects/07b861f95bf3498d990785ba72474d82/users/3f2a2bf913d1412f996acf4401ba7673/roles/9f07c51b3716428b89a49a756664305a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:22:11.514306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62a208-38cb-48d6-bbe0-6049b432ae97 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:11.575822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:11.590413 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0f373-2468-4b7e-a3b5-4b5f682ffa01 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:11.656897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57701814-34e8-478e-bb5c-1f1b99995ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:11.659164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57701814-34e8-478e-bb5c-1f1b99995ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:11.660132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57701814-34e8-478e-bb5c-1f1b99995ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:11.660814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57701814-34e8-478e-bb5c-1f1b99995ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:11.662397 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57701814-34e8-478e-bb5c-1f1b99995ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:11.663178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57701814-34e8-478e-bb5c-1f1b99995ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:11.665846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57701814-34e8-478e-bb5c-1f1b99995ec9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:11.681650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62a208-38cb-48d6-bbe0-6049b432ae97 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:11.689162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 290/586] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:11.691109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57701814-34e8-478e-bb5c-1f1b99995ec9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:11.725999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c0538-f318-4c49-9504-42719e3c0a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:11.726880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c0538-f318-4c49-9504-42719e3c0a5b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:11.727155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c0538-f318-4c49-9504-42719e3c0a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:11.727439 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c0538-f318-4c49-9504-42719e3c0a5b None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:11.728463 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c0538-f318-4c49-9504-42719e3c0a5b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:11.728722 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c0538-f318-4c49-9504-42719e3c0a5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:11.730994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c0538-f318-4c49-9504-42719e3c0a5b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:11.753911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:11.757768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 297/587] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Jun 7 08:22:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:11.758906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c0538-f318-4c49-9504-42719e3c0a5b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:11.900502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:11.909915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57701814-34e8-478e-bb5c-1f1b99995ec9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:11.931579 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c0538-f318-4c49-9504-42719e3c0a5b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:11.961460 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022982e6-cf36-41ab-9314-cfcc5826fd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:11.962446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022982e6-cf36-41ab-9314-cfcc5826fd0b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:11.962926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022982e6-cf36-41ab-9314-cfcc5826fd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:11.963700 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022982e6-cf36-41ab-9314-cfcc5826fd0b None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:11.964137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022982e6-cf36-41ab-9314-cfcc5826fd0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:11.964485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022982e6-cf36-41ab-9314-cfcc5826fd0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:11.966518 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022982e6-cf36-41ab-9314-cfcc5826fd0b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:11.984036 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 291/588] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:11.984967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022982e6-cf36-41ab-9314-cfcc5826fd0b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:11.987054 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbb924-eb59-4d33-b55a-f473c8290d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:11.990802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbb924-eb59-4d33-b55a-f473c8290d57 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:11.991738 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbb924-eb59-4d33-b55a-f473c8290d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:11.992528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbb924-eb59-4d33-b55a-f473c8290d57 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:12.013392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbb924-eb59-4d33-b55a-f473c8290d57 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:12.017945 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbb924-eb59-4d33-b55a-f473c8290d57 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-9ccbb924-eb59-4d33-b55a-f473c8290d57 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:12.037690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:12.039429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:12.041754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:12.041754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:12.041754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:12.041754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:12.041754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:12.041754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:12.041754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:12.041754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:12.041754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 08:22:12.041754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:12.041754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 298/589] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:12.041754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbb924-eb59-4d33-b55a-f473c8290d57 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:12.158465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:12.177186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022982e6-cf36-41ab-9314-cfcc5826fd0b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:12.216553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbb924-eb59-4d33-b55a-f473c8290d57 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:12.249716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54162ea-5d83-4b35-a810-166e138e1929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:12.250779 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54162ea-5d83-4b35-a810-166e138e1929 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:12.250779 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54162ea-5d83-4b35-a810-166e138e1929 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:12.251062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54162ea-5d83-4b35-a810-166e138e1929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:12.263671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54162ea-5d83-4b35-a810-166e138e1929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:12.265240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54162ea-5d83-4b35-a810-166e138e1929 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:12.290530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693e832-c1cb-43fd-b61a-d3b1f3315828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:12.291588 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693e832-c1cb-43fd-b61a-d3b1f3315828 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:12.291880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693e832-c1cb-43fd-b61a-d3b1f3315828 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:12.292214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693e832-c1cb-43fd-b61a-d3b1f3315828 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-a54162ea-5d83-4b35-a810-166e138e1929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:12.295026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:12.295927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:12.296825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:12.296825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:12.296825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:12.296825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:12.300249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 292/590] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:12.302021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54162ea-5d83-4b35-a810-166e138e1929 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:12.305037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693e832-c1cb-43fd-b61a-d3b1f3315828 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:12.307283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693e832-c1cb-43fd-b61a-d3b1f3315828 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-2693e832-c1cb-43fd-b61a-d3b1f3315828 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:12.327573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:12.328822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:12.329813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:12.329813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:12.329813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:12.329813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:12.329813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:12.329813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:12.329813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:12.329813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:12.329813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 08:22:12.329813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:12.336479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 299/591] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:12.336479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693e832-c1cb-43fd-b61a-d3b1f3315828 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:12.348971 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:12.469941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693e832-c1cb-43fd-b61a-d3b1f3315828 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:12.507543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54162ea-5d83-4b35-a810-166e138e1929 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:12.522933 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3ad55-4c85-43de-89d0-b09b8a88e0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:12.524257 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3ad55-4c85-43de-89d0-b09b8a88e0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:12.524957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3ad55-4c85-43de-89d0-b09b8a88e0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:12.525480 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3ad55-4c85-43de-89d0-b09b8a88e0b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:12.527076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3ad55-4c85-43de-89d0-b09b8a88e0b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:12.527867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3ad55-4c85-43de-89d0-b09b8a88e0b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:12.529765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3ad55-4c85-43de-89d0-b09b8a88e0b5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:12.549302 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 300/592] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:12.550279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3ad55-4c85-43de-89d0-b09b8a88e0b5 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:12.561378 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40824f61-a44c-4612-82c3-8409c587cc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:12.562391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40824f61-a44c-4612-82c3-8409c587cc87 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:12.562943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40824f61-a44c-4612-82c3-8409c587cc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:12.563908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40824f61-a44c-4612-82c3-8409c587cc87 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:12.565426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40824f61-a44c-4612-82c3-8409c587cc87 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:12.565943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40824f61-a44c-4612-82c3-8409c587cc87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:12.568089 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40824f61-a44c-4612-82c3-8409c587cc87 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:12.599510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 293/593] 10.222.0.39 () {56 vars in 1139 bytes} [Tue Jun 7 08:22:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:12.600679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40824f61-a44c-4612-82c3-8409c587cc87 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:12.603719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:12.672830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:12.737689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3ad55-4c85-43de-89d0-b09b8a88e0b5 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:12.740948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40824f61-a44c-4612-82c3-8409c587cc87 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:12.787390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2280e2-05ae-4907-8b5f-8d22d955b861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:12.788999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2280e2-05ae-4907-8b5f-8d22d955b861 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:12.789516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2280e2-05ae-4907-8b5f-8d22d955b861 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:12.789989 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2280e2-05ae-4907-8b5f-8d22d955b861 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:12.799508 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2280e2-05ae-4907-8b5f-8d22d955b861 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:12.801545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2280e2-05ae-4907-8b5f-8d22d955b861 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:12.803000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fd4e5-2dc1-4534-8ec7-8b7bbbb3025f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:12.804191 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd4e5-2dc1-4534-8ec7-8b7bbbb3025f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:12.804716 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd4e5-2dc1-4534-8ec7-8b7bbbb3025f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:12.805252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd4e5-2dc1-4534-8ec7-8b7bbbb3025f None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:12.806854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fd4e5-2dc1-4534-8ec7-8b7bbbb3025f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:12.807413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fd4e5-2dc1-4534-8ec7-8b7bbbb3025f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:12.812787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fd4e5-2dc1-4534-8ec7-8b7bbbb3025f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-8f2280e2-05ae-4907-8b5f-8d22d955b861 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:12.820568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:12.821657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:12.822553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:12.822553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:12.822553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:12.822553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:12.822553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:12.822553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:12.822553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:12.822553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:12.825292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 294/594] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:12.826368 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2280e2-05ae-4907-8b5f-8d22d955b861 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:12.857781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 301/595] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Jun 7 08:22:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:12.859375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fd4e5-2dc1-4534-8ec7-8b7bbbb3025f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:12.914251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:12.970181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2280e2-05ae-4907-8b5f-8d22d955b861 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:13.014322 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f2944-ecfe-4e86-a6e4-ad572c08382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:13.016313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f2944-ecfe-4e86-a6e4-ad572c08382f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:13.016847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f2944-ecfe-4e86-a6e4-ad572c08382f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:13.017344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f2944-ecfe-4e86-a6e4-ad572c08382f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:13.032526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f2944-ecfe-4e86-a6e4-ad572c08382f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:13.034701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f2944-ecfe-4e86-a6e4-ad572c08382f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-766f2944-ecfe-4e86-a6e4-ad572c08382f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:13.062241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:13.063222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:13.064282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:13.064282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:13.064282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:13.064282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:13.065153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fd4e5-2dc1-4534-8ec7-8b7bbbb3025f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:13.069614 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 295/596] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:13.071092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f2944-ecfe-4e86-a6e4-ad572c08382f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:13.098499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:13.110542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca4489-b21b-49b4-9ece-add6195a97b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:13.111521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca4489-b21b-49b4-9ece-add6195a97b0 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:13.112178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca4489-b21b-49b4-9ece-add6195a97b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:13.112663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca4489-b21b-49b4-9ece-add6195a97b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:13.113691 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca4489-b21b-49b4-9ece-add6195a97b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:13.114523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca4489-b21b-49b4-9ece-add6195a97b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:13.116198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca4489-b21b-49b4-9ece-add6195a97b0 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:13.132072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 302/597] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:13.133288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca4489-b21b-49b4-9ece-add6195a97b0 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:13.275371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f2944-ecfe-4e86-a6e4-ad572c08382f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:13.306914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:13.319045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca4489-b21b-49b4-9ece-add6195a97b0 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:13.350197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596bf09-922a-4f78-a75d-1cba72fadb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:13.352351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596bf09-922a-4f78-a75d-1cba72fadb21 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:13.352351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596bf09-922a-4f78-a75d-1cba72fadb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:13.352762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596bf09-922a-4f78-a75d-1cba72fadb21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:13.374699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f596bf09-922a-4f78-a75d-1cba72fadb21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:13.377255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f596bf09-922a-4f78-a75d-1cba72fadb21 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:13.397440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ed353-3400-44a0-b2ad-edfe51c1b264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:13.399018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ed353-3400-44a0-b2ad-edfe51c1b264 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:13.399792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ed353-3400-44a0-b2ad-edfe51c1b264 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:13.400513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ed353-3400-44a0-b2ad-edfe51c1b264 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:13.402126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ed353-3400-44a0-b2ad-edfe51c1b264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:13.405235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ed353-3400-44a0-b2ad-edfe51c1b264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:13.408143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ed353-3400-44a0-b2ad-edfe51c1b264 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f596bf09-922a-4f78-a75d-1cba72fadb21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:13.411702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:13.413455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:13.415140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:13.415140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:13.415140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:13.415140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:13.419024 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 296/598] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:13.419908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596bf09-922a-4f78-a75d-1cba72fadb21 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:13.441148 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 303/599] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:13.442247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ed353-3400-44a0-b2ad-edfe51c1b264 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:13.525288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:13.607484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596bf09-922a-4f78-a75d-1cba72fadb21 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:13.620000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ed353-3400-44a0-b2ad-edfe51c1b264 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:13.648744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcfd12-92e5-444b-9611-21d021ad21d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:13.650050 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcfd12-92e5-444b-9611-21d021ad21d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:13.650630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcfd12-92e5-444b-9611-21d021ad21d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:13.651265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcfd12-92e5-444b-9611-21d021ad21d3 None admin] PATH_INFO: `/v3/projects/07b861f95bf3498d990785ba72474d82/users/2b6943045d594d729fc7c4b15cbaa07e/roles/9f07c51b3716428b89a49a756664305a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:13.671273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcfd12-92e5-444b-9611-21d021ad21d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=07b861f95bf3498d990785ba72474d82, user_id=2b6943045d594d729fc7c4b15cbaa07e, role_id=9f07c51b3716428b89a49a756664305a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:13.673025 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcfd12-92e5-444b-9611-21d021ad21d3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:13.689131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af834d-b1ae-4c6d-b101-19ac8ddcd5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:13.689933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 297/600] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:22:13 2022] PUT /identity/v3/projects/07b861f95bf3498d990785ba72474d82/users/2b6943045d594d729fc7c4b15cbaa07e/roles/9f07c51b3716428b89a49a756664305a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:22:13.690509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af834d-b1ae-4c6d-b101-19ac8ddcd5bf None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:13.691071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af834d-b1ae-4c6d-b101-19ac8ddcd5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:13.691628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af834d-b1ae-4c6d-b101-19ac8ddcd5bf None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:13.691879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcfd12-92e5-444b-9611-21d021ad21d3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:13.693220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af834d-b1ae-4c6d-b101-19ac8ddcd5bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:13.693853 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af834d-b1ae-4c6d-b101-19ac8ddcd5bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:13.696619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af834d-b1ae-4c6d-b101-19ac8ddcd5bf None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:13.716045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:13.723834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 304/601] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:13.727224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af834d-b1ae-4c6d-b101-19ac8ddcd5bf None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:13.838061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcfd12-92e5-444b-9611-21d021ad21d3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:13.845988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af834d-b1ae-4c6d-b101-19ac8ddcd5bf None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:13.854234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:13.887875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ccfd7-fa6a-4432-af17-3efa73de26cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:13.889134 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ccfd7-fa6a-4432-af17-3efa73de26cf None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:13.889487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ccfd7-fa6a-4432-af17-3efa73de26cf None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:13.889763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ccfd7-fa6a-4432-af17-3efa73de26cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:13.892430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a926ea-f013-42f6-b149-8dc68ce056fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:13.894541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a926ea-f013-42f6-b149-8dc68ce056fc None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:13.895324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a926ea-f013-42f6-b149-8dc68ce056fc None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:13.895898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a926ea-f013-42f6-b149-8dc68ce056fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:13.899401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ccfd7-fa6a-4432-af17-3efa73de26cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:13.902879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ccfd7-fa6a-4432-af17-3efa73de26cf None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:13.906596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a926ea-f013-42f6-b149-8dc68ce056fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:13.908521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a926ea-f013-42f6-b149-8dc68ce056fc None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-e89ccfd7-fa6a-4432-af17-3efa73de26cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:13.918901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:13.920154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:13.921068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:13.921068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:13.921068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:13.921068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:13.921068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:13.921068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:13.921068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:13.921068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:13.921068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 305/602] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:13.926632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ccfd7-fa6a-4432-af17-3efa73de26cf None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-07a926ea-f013-42f6-b149-8dc68ce056fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:13.931788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:13.932751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:13.934406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:13.934406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:13.934406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:13.934406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:13.934406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:13.934406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:13.934406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:13.934406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:13.937558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 298/603] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:13.944252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a926ea-f013-42f6-b149-8dc68ce056fc None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:14.044033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ccfd7-fa6a-4432-af17-3efa73de26cf None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:14.053393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:14.084822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c471f-044d-490c-9073-88e4b1f607d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:14.085539 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c471f-044d-490c-9073-88e4b1f607d1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:14.085826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c471f-044d-490c-9073-88e4b1f607d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:14.086051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a926ea-f013-42f6-b149-8dc68ce056fc None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:14.086166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c471f-044d-490c-9073-88e4b1f607d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:14.086885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c471f-044d-490c-9073-88e4b1f607d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:14.087200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c471f-044d-490c-9073-88e4b1f607d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:14.088644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c471f-044d-490c-9073-88e4b1f607d1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:14.103301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 306/604] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:14.103515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e31588-fc29-42fb-9208-9ef7a4b854d6 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:14.103890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e31588-fc29-42fb-9208-9ef7a4b854d6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:14.104068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e31588-fc29-42fb-9208-9ef7a4b854d6 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:14.106351 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 307/605] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:14.114579 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e31588-fc29-42fb-9208-9ef7a4b854d6 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:14.131571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda89687-7058-4026-8407-a872c4218077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:14.132261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda89687-7058-4026-8407-a872c4218077 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:14.132572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda89687-7058-4026-8407-a872c4218077 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:14.132854 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda89687-7058-4026-8407-a872c4218077 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:14.133698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda89687-7058-4026-8407-a872c4218077 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:14.134613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda89687-7058-4026-8407-a872c4218077 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:14.136305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda89687-7058-4026-8407-a872c4218077 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:14.150139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 299/606] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:14.151247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6588b-2c3e-4f09-9ba8-3df935d06694 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:14.151536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6588b-2c3e-4f09-9ba8-3df935d06694 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:14.151812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6588b-2c3e-4f09-9ba8-3df935d06694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:14.159239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:14.181248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d6e6588b-2c3e-4f09-9ba8-3df935d06694 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:14.183172 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d6e6588b-2c3e-4f09-9ba8-3df935d06694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:14.272907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e31588-fc29-42fb-9208-9ef7a4b854d6 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:14.280346 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 300/607] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:14.281358 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6588b-2c3e-4f09-9ba8-3df935d06694 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:14.329270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45caa7-cd59-47d4-b03c-8ebf0000dc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:14.331797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45caa7-cd59-47d4-b03c-8ebf0000dc6e None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:14.332466 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45caa7-cd59-47d4-b03c-8ebf0000dc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:14.332990 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45caa7-cd59-47d4-b03c-8ebf0000dc6e None admin] PATH_INFO: `/v3/projects/07b861f95bf3498d990785ba72474d82/users/d1e4531ac696445ba5bfa9870984befd/roles/9f07c51b3716428b89a49a756664305a` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:14.352353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45caa7-cd59-47d4-b03c-8ebf0000dc6e None admin] RBAC: Authorizing `identity:create_grant(project_id=07b861f95bf3498d990785ba72474d82, user_id=d1e4531ac696445ba5bfa9870984befd, role_id=9f07c51b3716428b89a49a756664305a)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:14.354460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45caa7-cd59-47d4-b03c-8ebf0000dc6e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:14.360050 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:14.376982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 308/608] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:22:14 2022] PUT /identity/v3/projects/07b861f95bf3498d990785ba72474d82/users/d1e4531ac696445ba5bfa9870984befd/roles/9f07c51b3716428b89a49a756664305a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:22:14.378772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45caa7-cd59-47d4-b03c-8ebf0000dc6e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:14.409663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6588b-2c3e-4f09-9ba8-3df935d06694 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:14.453057 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a11af-bbee-4628-a2a9-23a9d21646db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:14.454894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a11af-bbee-4628-a2a9-23a9d21646db None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:14.454894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a11af-bbee-4628-a2a9-23a9d21646db None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:14.455183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a11af-bbee-4628-a2a9-23a9d21646db None admin] PATH_INFO: `/v3/projects/07b861f95bf3498d990785ba72474d82/users/7ddf89deb28749eda2d26ad66bc282f8/roles/9f07c51b3716428b89a49a756664305a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:14.473926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a11af-bbee-4628-a2a9-23a9d21646db None admin] RBAC: Authorizing `identity:create_grant(project_id=07b861f95bf3498d990785ba72474d82, user_id=7ddf89deb28749eda2d26ad66bc282f8, role_id=9f07c51b3716428b89a49a756664305a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:14.475305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a11af-bbee-4628-a2a9-23a9d21646db None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:14.507359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 301/609] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:22:14 2022] PUT /identity/v3/projects/07b861f95bf3498d990785ba72474d82/users/7ddf89deb28749eda2d26ad66bc282f8/roles/9f07c51b3716428b89a49a756664305a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:22:14.510535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad3e0-89c0-4dd5-8ebe-389f851dcce1 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:14.510843 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad3e0-89c0-4dd5-8ebe-389f851dcce1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:14.511104 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad3e0-89c0-4dd5-8ebe-389f851dcce1 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:14.515013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 302/610] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:14.522775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:14.525694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e5bbe-4b7d-45f3-830f-ae86b7d76d8b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:14.526407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e5bbe-4b7d-45f3-830f-ae86b7d76d8b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:14.526887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e5bbe-4b7d-45f3-830f-ae86b7d76d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:14.551964 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-eb1e5bbe-4b7d-45f3-830f-ae86b7d76d8b None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:14.553839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-eb1e5bbe-4b7d-45f3-830f-ae86b7d76d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:14.566337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45caa7-cd59-47d4-b03c-8ebf0000dc6e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:14.623203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467560e0-cb8c-49b3-8e56-4f231380b7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:14.624092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467560e0-cb8c-49b3-8e56-4f231380b7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:14.624380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467560e0-cb8c-49b3-8e56-4f231380b7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:14.624658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467560e0-cb8c-49b3-8e56-4f231380b7d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:14.643105 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467560e0-cb8c-49b3-8e56-4f231380b7d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-467560e0-cb8c-49b3-8e56-4f231380b7d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:14.651983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:14.653075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:14.653941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:14.653941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:14.653941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:14.653941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:14.653941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:14.653941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:14.653941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:14.653941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:14.653941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:14.653941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:14.653941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:14.653941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:14.654647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 303/611] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:14.661214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 309/612] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:14.664273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e5bbe-4b7d-45f3-830f-ae86b7d76d8b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:14.683127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467560e0-cb8c-49b3-8e56-4f231380b7d9 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:14.742971 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:14.843370 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e5bbe-4b7d-45f3-830f-ae86b7d76d8b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:14.848013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467560e0-cb8c-49b3-8e56-4f231380b7d9 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:14.899939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3467f-0fb0-4315-b9af-403d75104e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:14.901247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3467f-0fb0-4315-b9af-403d75104e10 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:14.901759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3467f-0fb0-4315-b9af-403d75104e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:14.902207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3467f-0fb0-4315-b9af-403d75104e10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:14.902942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf519a-2e83-4a39-8313-c2830164e4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:14.905634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf519a-2e83-4a39-8313-c2830164e4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:14.905634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf519a-2e83-4a39-8313-c2830164e4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:14.905634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf519a-2e83-4a39-8313-c2830164e4a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:14.907090 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf519a-2e83-4a39-8313-c2830164e4a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:14.907746 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf519a-2e83-4a39-8313-c2830164e4a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:14.912938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf519a-2e83-4a39-8313-c2830164e4a4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:14.917526 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:14.922222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3467f-0fb0-4315-b9af-403d75104e10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:14.929974 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 304/613] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-b0e3467f-0fb0-4315-b9af-403d75104e10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:14.937378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:14.938583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:14.939608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:14.939608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:14.939608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:14.939608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:14.939608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:14.939608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:14.939608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:14.939608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:14.939608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:14.939608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:14.939608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:14.939608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:14.944081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 310/614] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:14.946484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf519a-2e83-4a39-8313-c2830164e4a4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:14.951045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3467f-0fb0-4315-b9af-403d75104e10 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:15.198365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf519a-2e83-4a39-8313-c2830164e4a4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:15.211920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3467f-0fb0-4315-b9af-403d75104e10 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:15.216468 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:15.258633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f8c8b-88a3-453c-9a7a-ca1b372638e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:15.259621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f8c8b-88a3-453c-9a7a-ca1b372638e5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:15.260007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f8c8b-88a3-453c-9a7a-ca1b372638e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:15.260290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f8c8b-88a3-453c-9a7a-ca1b372638e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:15.280837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f8c8b-88a3-453c-9a7a-ca1b372638e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:15.282775 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f8c8b-88a3-453c-9a7a-ca1b372638e5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-515f8c8b-88a3-453c-9a7a-ca1b372638e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:15.316699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:15.317668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:15.318546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:15.318546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:15.318546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:15.318546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:15.318546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 305/615] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:15.327271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f8c8b-88a3-453c-9a7a-ca1b372638e5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:15.347696 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a448145-38d7-42f8-8f51-dad8a747591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:15.349132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a448145-38d7-42f8-8f51-dad8a747591e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:15.351149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a448145-38d7-42f8-8f51-dad8a747591e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:15.351649 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a448145-38d7-42f8-8f51-dad8a747591e None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:15.352892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a448145-38d7-42f8-8f51-dad8a747591e None admin] RBAC: Adding query filter params (name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:15.354910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a448145-38d7-42f8-8f51-dad8a747591e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:15.356637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a448145-38d7-42f8-8f51-dad8a747591e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:15.378258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 311/616] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:15.383973 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a448145-38d7-42f8-8f51-dad8a747591e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:15.471855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f8c8b-88a3-453c-9a7a-ca1b372638e5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:15.525916 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef2190-378d-4105-a028-352d580dd399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:15.526620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef2190-378d-4105-a028-352d580dd399 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:15.526912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef2190-378d-4105-a028-352d580dd399 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:15.527191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef2190-378d-4105-a028-352d580dd399 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:15.528014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef2190-378d-4105-a028-352d580dd399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:15.528319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef2190-378d-4105-a028-352d580dd399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:15.529340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:15.530088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef2190-378d-4105-a028-352d580dd399 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:15.561229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 306/617] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:15.568900 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef2190-378d-4105-a028-352d580dd399 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:15.589294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a448145-38d7-42f8-8f51-dad8a747591e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:15.625657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:15.630989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ed385-b99e-4b6a-bcaf-f3b103118a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:15.631834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ed385-b99e-4b6a-bcaf-f3b103118a8f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:15.632120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ed385-b99e-4b6a-bcaf-f3b103118a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:15.632393 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ed385-b99e-4b6a-bcaf-f3b103118a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:15.655389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ed385-b99e-4b6a-bcaf-f3b103118a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:15.656884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ed385-b99e-4b6a-bcaf-f3b103118a8f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-4b0ed385-b99e-4b6a-bcaf-f3b103118a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:15.689815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:15.690776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:15.691792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:15.691792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:15.691792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:15.691792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:15.691792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 312/618] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:15.696657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ed385-b99e-4b6a-bcaf-f3b103118a8f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:15.797427 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef2190-378d-4105-a028-352d580dd399 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:15.836898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d8802-3d31-4982-af42-9078d57adceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:15.837973 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d8802-3d31-4982-af42-9078d57adceb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:15.838251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d8802-3d31-4982-af42-9078d57adceb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:15.838534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d8802-3d31-4982-af42-9078d57adceb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:15.848403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d8802-3d31-4982-af42-9078d57adceb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:15.849887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d8802-3d31-4982-af42-9078d57adceb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-7d4d8802-3d31-4982-af42-9078d57adceb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:15.867826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:15.868857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:15.869835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:15.869835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:15.869835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:15.869835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:15.869835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:15.869835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:15.869835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:15.869835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:15.869835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:22:15.869835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:15.869835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 307/619] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:15.876432 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d8802-3d31-4982-af42-9078d57adceb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:15.919656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ed385-b99e-4b6a-bcaf-f3b103118a8f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:15.947082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:16.000750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:16.013706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d8802-3d31-4982-af42-9078d57adceb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:16.065920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d23ee-fd07-47c5-a15c-f69415c36b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:16.066520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d23ee-fd07-47c5-a15c-f69415c36b14 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:16.066806 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d23ee-fd07-47c5-a15c-f69415c36b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:16.067081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d23ee-fd07-47c5-a15c-f69415c36b14 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:16.067815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d23ee-fd07-47c5-a15c-f69415c36b14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:16.068123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d23ee-fd07-47c5-a15c-f69415c36b14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:16.068636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146e9ff-8ff8-4dce-80c0-268a0f0ec3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:16.069342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146e9ff-8ff8-4dce-80c0-268a0f0ec3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:16.069541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d23ee-fd07-47c5-a15c-f69415c36b14 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:16.069706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146e9ff-8ff8-4dce-80c0-268a0f0ec3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:16.069961 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146e9ff-8ff8-4dce-80c0-268a0f0ec3d1 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:16.070879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146e9ff-8ff8-4dce-80c0-268a0f0ec3d1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:16.071516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146e9ff-8ff8-4dce-80c0-268a0f0ec3d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:16.073064 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146e9ff-8ff8-4dce-80c0-268a0f0ec3d1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:16.096887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 308/620] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Jun 7 08:22:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:16.102743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146e9ff-8ff8-4dce-80c0-268a0f0ec3d1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:16.111505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 313/621] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:16.134238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d23ee-fd07-47c5-a15c-f69415c36b14 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:16.249922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146e9ff-8ff8-4dce-80c0-268a0f0ec3d1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:16.306343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65451697-e341-4b8d-83ca-d2503fe7cfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:16.307203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65451697-e341-4b8d-83ca-d2503fe7cfbf None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:16.307575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65451697-e341-4b8d-83ca-d2503fe7cfbf None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:16.307857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65451697-e341-4b8d-83ca-d2503fe7cfbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:16.317595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:16.321730 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65451697-e341-4b8d-83ca-d2503fe7cfbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:16.323570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65451697-e341-4b8d-83ca-d2503fe7cfbf None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-65451697-e341-4b8d-83ca-d2503fe7cfbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:16.349116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:16.350076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:16.351424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:16.351424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:16.351424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:16.351424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:16.351424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 309/622] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:16.358948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65451697-e341-4b8d-83ca-d2503fe7cfbf None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:16.491142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d23ee-fd07-47c5-a15c-f69415c36b14 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:16.567010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65451697-e341-4b8d-83ca-d2503fe7cfbf None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:16.615152 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09775e-3b93-43a3-88ee-8ba912a084b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:16.616990 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09775e-3b93-43a3-88ee-8ba912a084b6 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:16.617602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09775e-3b93-43a3-88ee-8ba912a084b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:16.618167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09775e-3b93-43a3-88ee-8ba912a084b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:16.618946 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765e6ac-ec68-4037-bcb6-68fcc757d571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:16.619737 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765e6ac-ec68-4037-bcb6-68fcc757d571 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:16.620123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765e6ac-ec68-4037-bcb6-68fcc757d571 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:16.620497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765e6ac-ec68-4037-bcb6-68fcc757d571 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:16.621897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765e6ac-ec68-4037-bcb6-68fcc757d571 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:16.622436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765e6ac-ec68-4037-bcb6-68fcc757d571 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:16.624021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765e6ac-ec68-4037-bcb6-68fcc757d571 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:16.636232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09775e-3b93-43a3-88ee-8ba912a084b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:16.638248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 310/623] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:16.639178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09775e-3b93-43a3-88ee-8ba912a084b6 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:16.639793 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16974f8e-b3cc-4df2-9bb0-a9b4fa836ec5 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:16.640201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16974f8e-b3cc-4df2-9bb0-a9b4fa836ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:16.640578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16974f8e-b3cc-4df2-9bb0-a9b4fa836ec5 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:16.646348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 311/624] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:16.647908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e955-5f93-44b1-974e-21da60c48ecb None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:16.648310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e955-5f93-44b1-974e-21da60c48ecb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:16.648688 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e955-5f93-44b1-974e-21da60c48ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:16.669984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2821e955-5f93-44b1-974e-21da60c48ecb None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:16.671355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2821e955-5f93-44b1-974e-21da60c48ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-2d09775e-3b93-43a3-88ee-8ba912a084b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:16.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:16.686029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:16.687193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:16.687193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:16.687193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:16.687193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:16.687193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:16.687193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:16.687193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:16.687193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:16.687193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:22:16.687193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:16.687918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 314/625] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jun 7 08:22:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 554 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:16.689928 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09775e-3b93-43a3-88ee-8ba912a084b6 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:16.749321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:16.750268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:16.775016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 312/626] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:22:16.782146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821e955-5f93-44b1-974e-21da60c48ecb None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:16.834964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09775e-3b93-43a3-88ee-8ba912a084b6 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:16.876873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf5522-d42c-4457-8ef8-ec4e4a8d2140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:16.877631 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf5522-d42c-4457-8ef8-ec4e4a8d2140 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:16.877917 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf5522-d42c-4457-8ef8-ec4e4a8d2140 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:16.878190 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf5522-d42c-4457-8ef8-ec4e4a8d2140 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:16.890577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf5522-d42c-4457-8ef8-ec4e4a8d2140 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:16.890577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf5522-d42c-4457-8ef8-ec4e4a8d2140 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-90bf5522-d42c-4457-8ef8-ec4e4a8d2140 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:16.906949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:16.907924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:16.908803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:16.908803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:16.908803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:16.908803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:16.908803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:16.908803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:16.908803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:16.908803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:16.909820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 315/627] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:16.916816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf5522-d42c-4457-8ef8-ec4e4a8d2140 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:16.920065 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821e955-5f93-44b1-974e-21da60c48ecb None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:16.960547 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0c073-6a59-45eb-828f-009027c224bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:16.961347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0c073-6a59-45eb-828f-009027c224bc None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:16.961635 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0c073-6a59-45eb-828f-009027c224bc None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:16.961911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0c073-6a59-45eb-828f-009027c224bc None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:16.962767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0c073-6a59-45eb-828f-009027c224bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:16.963078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0c073-6a59-45eb-828f-009027c224bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:16.964533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0c073-6a59-45eb-828f-009027c224bc None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:16.985016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 313/628] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jun 7 08:22:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:16.985919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0c073-6a59-45eb-828f-009027c224bc None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:17.051637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:17.062368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:17.085331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf5522-d42c-4457-8ef8-ec4e4a8d2140 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:17.125348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef9256-d84c-4e38-80e3-2342cc361de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:17.126570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef9256-d84c-4e38-80e3-2342cc361de3 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:17.127132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef9256-d84c-4e38-80e3-2342cc361de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:17.127564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef9256-d84c-4e38-80e3-2342cc361de3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:17.141667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef9256-d84c-4e38-80e3-2342cc361de3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-12ef9256-d84c-4e38-80e3-2342cc361de3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:17.150231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:17.151451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:17.153110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:17.153110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:17.153110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:17.153110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:17.153110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:17.153110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:17.153110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:17.153110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:17.153110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:17.153110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:17.153110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:17.153110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:17.155840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 316/629] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:17.156909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef9256-d84c-4e38-80e3-2342cc361de3 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:17.181029 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0c073-6a59-45eb-828f-009027c224bc None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:17.245944 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f437eb0-a5c2-4f31-a54d-f65e8baca0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:17.247138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f437eb0-a5c2-4f31-a54d-f65e8baca0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:17.247632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f437eb0-a5c2-4f31-a54d-f65e8baca0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:17.248114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f437eb0-a5c2-4f31-a54d-f65e8baca0b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:17.249632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f437eb0-a5c2-4f31-a54d-f65e8baca0b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:17.250197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f437eb0-a5c2-4f31-a54d-f65e8baca0b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:17.252909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f437eb0-a5c2-4f31-a54d-f65e8baca0b3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:17.280284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 314/630] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:17.280509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef9256-d84c-4e38-80e3-2342cc361de3 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:17.280623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f437eb0-a5c2-4f31-a54d-f65e8baca0b3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:17.320002 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898c2af-626b-45d9-91d4-6c654b65fe2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:17.320665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898c2af-626b-45d9-91d4-6c654b65fe2e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:17.320949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898c2af-626b-45d9-91d4-6c654b65fe2e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:17.321343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898c2af-626b-45d9-91d4-6c654b65fe2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:17.342821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:17.392960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:17.397938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f437eb0-a5c2-4f31-a54d-f65e8baca0b3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:17.400448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898c2af-626b-45d9-91d4-6c654b65fe2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:17.402040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898c2af-626b-45d9-91d4-6c654b65fe2e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:17.440478 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816a9af-f14b-4fba-a436-f9a2d20204bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:17.441191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816a9af-f14b-4fba-a436-f9a2d20204bb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:17.441470 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816a9af-f14b-4fba-a436-f9a2d20204bb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:17.441742 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816a9af-f14b-4fba-a436-f9a2d20204bb None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:17.442501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816a9af-f14b-4fba-a436-f9a2d20204bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:17.442802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816a9af-f14b-4fba-a436-f9a2d20204bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:17.444183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816a9af-f14b-4fba-a436-f9a2d20204bb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:17.461265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 315/631] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:17.461265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4369144-a4a3-4e93-8884-9cdc78fa9b7f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:17.461265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4369144-a4a3-4e93-8884-9cdc78fa9b7f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:17.461265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4369144-a4a3-4e93-8884-9cdc78fa9b7f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:17.461265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 316/632] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:17.461265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4369144-a4a3-4e93-8884-9cdc78fa9b7f None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:17.524345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 317/633] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:22:17.525338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898c2af-626b-45d9-91d4-6c654b65fe2e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:17.583318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:17.584359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4369144-a4a3-4e93-8884-9cdc78fa9b7f None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:17.624469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0ee86-60f5-41bf-8da5-f290cf832ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:17.625382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0ee86-60f5-41bf-8da5-f290cf832ac8 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:17.625665 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0ee86-60f5-41bf-8da5-f290cf832ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:17.625938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0ee86-60f5-41bf-8da5-f290cf832ac8 None admin] PATH_INFO: `/v3/projects/07b861f95bf3498d990785ba72474d82/users/586ddffa77a54521b42b2aa50f16cb92/roles/9f07c51b3716428b89a49a756664305a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:17.636347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898c2af-626b-45d9-91d4-6c654b65fe2e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:17.649038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0ee86-60f5-41bf-8da5-f290cf832ac8 None admin] RBAC: Authorizing `identity:create_grant(project_id=07b861f95bf3498d990785ba72474d82, user_id=586ddffa77a54521b42b2aa50f16cb92, role_id=9f07c51b3716428b89a49a756664305a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:17.651574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0ee86-60f5-41bf-8da5-f290cf832ac8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:17.670984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 317/634] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:22:17 2022] PUT /identity/v3/projects/07b861f95bf3498d990785ba72474d82/users/586ddffa77a54521b42b2aa50f16cb92/roles/9f07c51b3716428b89a49a756664305a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:22:17.673455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f693e-0da0-43e8-bcb3-aa5434effafc None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:17.673905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f693e-0da0-43e8-bcb3-aa5434effafc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:17.674373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f693e-0da0-43e8-bcb3-aa5434effafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:17.676932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414cc41-6df8-427a-acb2-7d87d2b59f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:17.678041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414cc41-6df8-427a-acb2-7d87d2b59f4e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:17.678518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414cc41-6df8-427a-acb2-7d87d2b59f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:17.678973 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414cc41-6df8-427a-acb2-7d87d2b59f4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:17.693753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414cc41-6df8-427a-acb2-7d87d2b59f4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:17.695527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414cc41-6df8-427a-acb2-7d87d2b59f4e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:17.711259 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-af7f693e-0da0-43e8-bcb3-aa5434effafc None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:17.713146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-af7f693e-0da0-43e8-bcb3-aa5434effafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:17.718208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-3414cc41-6df8-427a-acb2-7d87d2b59f4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:17.718626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:17.728120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:17.730196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:17.730196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:17.730196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:17.730196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:17.730196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 318/635] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:17.730196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149cb21-7f53-4347-8e30-7d5fb3e398a6 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:17.730196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149cb21-7f53-4347-8e30-7d5fb3e398a6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:17.730196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149cb21-7f53-4347-8e30-7d5fb3e398a6 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:17.730196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 319/636] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:17.731894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149cb21-7f53-4347-8e30-7d5fb3e398a6 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:17.816247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 318/637] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:17.817321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f693e-0da0-43e8-bcb3-aa5434effafc None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:17.879888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149cb21-7f53-4347-8e30-7d5fb3e398a6 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:17.882656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:17.932209 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06d9c7-dccd-4c1d-8f14-9aa6a9d8a390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:17.933231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06d9c7-dccd-4c1d-8f14-9aa6a9d8a390 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:17.934116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06d9c7-dccd-4c1d-8f14-9aa6a9d8a390 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:17.934416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06d9c7-dccd-4c1d-8f14-9aa6a9d8a390 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:17.952264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06d9c7-dccd-4c1d-8f14-9aa6a9d8a390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:17.954030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06d9c7-dccd-4c1d-8f14-9aa6a9d8a390 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:17.964064 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f693e-0da0-43e8-bcb3-aa5434effafc None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-8a06d9c7-dccd-4c1d-8f14-9aa6a9d8a390 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:17.973890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:17.975640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:17.978888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:17.978888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:17.978888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:17.978888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:17.978888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 320/638] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:17.978888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7b255-20eb-4f6a-8582-e0e90e61da76 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:17.978888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7b255-20eb-4f6a-8582-e0e90e61da76 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:17.979662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7b255-20eb-4f6a-8582-e0e90e61da76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:17.998016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e7c7b255-20eb-4f6a-8582-e0e90e61da76 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:17.999228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e7c7b255-20eb-4f6a-8582-e0e90e61da76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:18.014866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43a0bd-c080-44bb-9010-76a5a85a1781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:18.015638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43a0bd-c080-44bb-9010-76a5a85a1781 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:18.015925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43a0bd-c080-44bb-9010-76a5a85a1781 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:18.016204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43a0bd-c080-44bb-9010-76a5a85a1781 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:18.016928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43a0bd-c080-44bb-9010-76a5a85a1781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:18.017301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43a0bd-c080-44bb-9010-76a5a85a1781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:18.018630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43a0bd-c080-44bb-9010-76a5a85a1781 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:18.036772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:18.040455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 319/639] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:18.041405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43a0bd-c080-44bb-9010-76a5a85a1781 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:18.107884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 321/640] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:18.108761 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7b255-20eb-4f6a-8582-e0e90e61da76 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:18.210765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43a0bd-c080-44bb-9010-76a5a85a1781 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:18.255115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:18.270159 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd15f91-222e-4f2a-aa88-6bf1a153e6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:18.271150 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd15f91-222e-4f2a-aa88-6bf1a153e6ed None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:18.271629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd15f91-222e-4f2a-aa88-6bf1a153e6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:18.272005 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd15f91-222e-4f2a-aa88-6bf1a153e6ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:18.282922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7b255-20eb-4f6a-8582-e0e90e61da76 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:18.287244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd15f91-222e-4f2a-aa88-6bf1a153e6ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-6fd15f91-222e-4f2a-aa88-6bf1a153e6ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:18.296105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:18.297566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:18.298968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:18.298968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:18.298968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:18.298968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:18.298968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:18.298968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:18.298968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:18.298968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:18.298968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:18.298968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:18.298968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:18.298968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:18.298968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 320/641] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:18.299809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd15f91-222e-4f2a-aa88-6bf1a153e6ed None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:18.345639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced1a3-2dc8-49bd-b3fc-9b4620a8a41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:18.346837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced1a3-2dc8-49bd-b3fc-9b4620a8a41a None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:18.347288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced1a3-2dc8-49bd-b3fc-9b4620a8a41a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:18.347757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced1a3-2dc8-49bd-b3fc-9b4620a8a41a None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:18.348713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eced1a3-2dc8-49bd-b3fc-9b4620a8a41a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:18.349207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eced1a3-2dc8-49bd-b3fc-9b4620a8a41a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:18.350994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eced1a3-2dc8-49bd-b3fc-9b4620a8a41a None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:18.372537 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 322/642] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:18.372537 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced1a3-2dc8-49bd-b3fc-9b4620a8a41a None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:18.407913 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:18.479240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd15f91-222e-4f2a-aa88-6bf1a153e6ed None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:18.511450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced1a3-2dc8-49bd-b3fc-9b4620a8a41a None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:18.519963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6f02-89eb-4465-91fc-e7a737aef71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:18.520779 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6f02-89eb-4465-91fc-e7a737aef71a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:18.521062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6f02-89eb-4465-91fc-e7a737aef71a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:18.521362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6f02-89eb-4465-91fc-e7a737aef71a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:18.530801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf6f02-89eb-4465-91fc-e7a737aef71a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:18.532308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf6f02-89eb-4465-91fc-e7a737aef71a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-fbdf6f02-89eb-4465-91fc-e7a737aef71a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:18.549395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:18.556752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:18.557978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:18.557978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:18.557978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:18.557978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:18.557978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:18.557978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:18.557978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:18.557978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:18.557978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:22:18.557978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:18.557978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 321/643] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:18.557978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6f02-89eb-4465-91fc-e7a737aef71a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:18.572521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3396fb-d7ad-4d03-aa66-1ee951a1b3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:18.573657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3396fb-d7ad-4d03-aa66-1ee951a1b3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:18.574704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3396fb-d7ad-4d03-aa66-1ee951a1b3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:18.576141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3396fb-d7ad-4d03-aa66-1ee951a1b3c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:18.599314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3396fb-d7ad-4d03-aa66-1ee951a1b3c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-ed3396fb-d7ad-4d03-aa66-1ee951a1b3c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:18.613328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:18.614359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:18.615243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:18.615243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:18.615243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:18.615243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:18.615243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:18.615243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:18.615243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:18.615243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:18.615243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:18.615243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:18.615243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:18.615243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:18.621921 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 323/644] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:18.621921 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3396fb-d7ad-4d03-aa66-1ee951a1b3c8 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:18.627416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:18.725317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6f02-89eb-4465-91fc-e7a737aef71a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:18.756505 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:18.777993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdfe6d-1c2a-474d-96ed-4ec6d7a7e491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:18.778782 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdfe6d-1c2a-474d-96ed-4ec6d7a7e491 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:18.779195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdfe6d-1c2a-474d-96ed-4ec6d7a7e491 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:18.779398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdfe6d-1c2a-474d-96ed-4ec6d7a7e491 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:18.780182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdfe6d-1c2a-474d-96ed-4ec6d7a7e491 None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:18.780484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdfe6d-1c2a-474d-96ed-4ec6d7a7e491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:18.782003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdfe6d-1c2a-474d-96ed-4ec6d7a7e491 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:18.788838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3396fb-d7ad-4d03-aa66-1ee951a1b3c8 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:18.791866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 322/645] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:18.794583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdfe6d-1c2a-474d-96ed-4ec6d7a7e491 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:18.850689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c3f6e-2a77-4060-b9ea-f6f83b34f4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:18.852041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c3f6e-2a77-4060-b9ea-f6f83b34f4dc None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:18.852478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c3f6e-2a77-4060-b9ea-f6f83b34f4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:18.852904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c3f6e-2a77-4060-b9ea-f6f83b34f4dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:18.865636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c3f6e-2a77-4060-b9ea-f6f83b34f4dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:18.868779 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c3f6e-2a77-4060-b9ea-f6f83b34f4dc None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-d29c3f6e-2a77-4060-b9ea-f6f83b34f4dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:18.900803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:18.902667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:18.908927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:18.908927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:18.908927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:18.908927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:18.908927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:18.908927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:18.908927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:18.908927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:18.908927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 324/646] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:18.908927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c3f6e-2a77-4060-b9ea-f6f83b34f4dc None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:18.981570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdfe6d-1c2a-474d-96ed-4ec6d7a7e491 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:19.011546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:19.031979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b5d33-6fcc-4591-a656-35be9c6663b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:19.033436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b5d33-6fcc-4591-a656-35be9c6663b5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:19.034726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b5d33-6fcc-4591-a656-35be9c6663b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:19.035430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b5d33-6fcc-4591-a656-35be9c6663b5 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:19.037176 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b5d33-6fcc-4591-a656-35be9c6663b5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:19.038395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b5d33-6fcc-4591-a656-35be9c6663b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:19.041596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b5d33-6fcc-4591-a656-35be9c6663b5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:19.064769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 323/647] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Jun 7 08:22:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:19.066023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b5d33-6fcc-4591-a656-35be9c6663b5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:19.082593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c3f6e-2a77-4060-b9ea-f6f83b34f4dc None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:19.123210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa31109-6c51-4bfe-b66c-2a06c284d571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:19.123890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa31109-6c51-4bfe-b66c-2a06c284d571 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:19.124171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa31109-6c51-4bfe-b66c-2a06c284d571 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:19.124447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa31109-6c51-4bfe-b66c-2a06c284d571 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:19.125247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa31109-6c51-4bfe-b66c-2a06c284d571 None admin] RBAC: Adding query filter params (name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:19.125549 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa31109-6c51-4bfe-b66c-2a06c284d571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:19.126947 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa31109-6c51-4bfe-b66c-2a06c284d571 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:19.142804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 325/648] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:19.142804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa31109-6c51-4bfe-b66c-2a06c284d571 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:19.147839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:19.250479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b5d33-6fcc-4591-a656-35be9c6663b5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:19.297591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa31109-6c51-4bfe-b66c-2a06c284d571 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:19.301598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadcacf-1186-4b8c-adcc-ca7f449035f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:19.303083 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadcacf-1186-4b8c-adcc-ca7f449035f2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:19.303705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadcacf-1186-4b8c-adcc-ca7f449035f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:19.304309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadcacf-1186-4b8c-adcc-ca7f449035f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:19.323352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadcacf-1186-4b8c-adcc-ca7f449035f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:19.326909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadcacf-1186-4b8c-adcc-ca7f449035f2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:19.332460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:19.343837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a9502-f41a-4645-aced-9c16b619f41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:19.344969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a9502-f41a-4645-aced-9c16b619f41e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:19.344969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a9502-f41a-4645-aced-9c16b619f41e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:19.345420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a9502-f41a-4645-aced-9c16b619f41e None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:19.346314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a9502-f41a-4645-aced-9c16b619f41e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:19.346699 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a9502-f41a-4645-aced-9c16b619f41e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:19.348251 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a9502-f41a-4645-aced-9c16b619f41e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:19.370372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 326/649] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-bdadcacf-1186-4b8c-adcc-ca7f449035f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:19.370638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:19.372754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:19.376862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:19.376862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:19.376862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:19.376862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:19.377349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a9502-f41a-4645-aced-9c16b619f41e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:19.380200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 324/650] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:19.380200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadcacf-1186-4b8c-adcc-ca7f449035f2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:19.512714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a9502-f41a-4645-aced-9c16b619f41e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:19.521046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:19.532556 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadcacf-1186-4b8c-adcc-ca7f449035f2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:19.572394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3b907-568e-40d2-9ab5-5b3caec06a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:19.573149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3b907-568e-40d2-9ab5-5b3caec06a31 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:19.573432 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3b907-568e-40d2-9ab5-5b3caec06a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:19.573703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3b907-568e-40d2-9ab5-5b3caec06a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:19.582509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317da9bb-0dfe-4480-a665-4174bc46c778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:19.583662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317da9bb-0dfe-4480-a665-4174bc46c778 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:19.584020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317da9bb-0dfe-4480-a665-4174bc46c778 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:19.584346 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317da9bb-0dfe-4480-a665-4174bc46c778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:19.588991 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3b907-568e-40d2-9ab5-5b3caec06a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:19.590544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3b907-568e-40d2-9ab5-5b3caec06a31 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-73a3b907-568e-40d2-9ab5-5b3caec06a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:19.610395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:19.611410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:19.618375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:19.618375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:19.618375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:19.618375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:19.618375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 325/651] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:19.618375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3b907-568e-40d2-9ab5-5b3caec06a31 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:19.694441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:19.713825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317da9bb-0dfe-4480-a665-4174bc46c778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:19.715867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317da9bb-0dfe-4480-a665-4174bc46c778 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:19.772172 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3b907-568e-40d2-9ab5-5b3caec06a31 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:19.817669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9965eaa-2347-4153-9d9c-06266d920a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:19.818468 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9965eaa-2347-4153-9d9c-06266d920a0c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:19.818760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9965eaa-2347-4153-9d9c-06266d920a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:19.819037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9965eaa-2347-4153-9d9c-06266d920a0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:19.819888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9965eaa-2347-4153-9d9c-06266d920a0c None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=3f2a2bf913d1412f996acf4401ba7673) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:19.820198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9965eaa-2347-4153-9d9c-06266d920a0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:19.821676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9965eaa-2347-4153-9d9c-06266d920a0c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:19.833743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 326/652] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:19 2022] GET /identity/v3/role_assignments?user.id=3f2a2bf913d1412f996acf4401ba7673&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:19.835335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9965eaa-2347-4153-9d9c-06266d920a0c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:19.861643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:19.882942 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 327/653] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:22:19.885247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317da9bb-0dfe-4480-a665-4174bc46c778 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:20.031171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9965eaa-2347-4153-9d9c-06266d920a0c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:20.059277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:20.085024 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317da9bb-0dfe-4480-a665-4174bc46c778 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:20.095277 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd0fed-b09f-45d5-a45b-66c6949282e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:20.096285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd0fed-b09f-45d5-a45b-66c6949282e6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:20.096565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd0fed-b09f-45d5-a45b-66c6949282e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:20.096834 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd0fed-b09f-45d5-a45b-66c6949282e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:20.097599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd0fed-b09f-45d5-a45b-66c6949282e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:20.097904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd0fed-b09f-45d5-a45b-66c6949282e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:20.099291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd0fed-b09f-45d5-a45b-66c6949282e6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:20.114257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 327/654] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:20.114257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd0fed-b09f-45d5-a45b-66c6949282e6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:20.137450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edc957-c0ca-45b3-bfe4-951585f84edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:20.139846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edc957-c0ca-45b3-bfe4-951585f84edd None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:20.139846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edc957-c0ca-45b3-bfe4-951585f84edd None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:20.139846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edc957-c0ca-45b3-bfe4-951585f84edd None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:20.140569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24edc957-c0ca-45b3-bfe4-951585f84edd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:20.141109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24edc957-c0ca-45b3-bfe4-951585f84edd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:20.143483 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24edc957-c0ca-45b3-bfe4-951585f84edd None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:20.166135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 328/655] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:20.167692 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3700f69-1d5a-4c4f-890c-df79ae81093c None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:20.168111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3700f69-1d5a-4c4f-890c-df79ae81093c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:20.168680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3700f69-1d5a-4c4f-890c-df79ae81093c None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:20.170702 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 329/656] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:20.171550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3700f69-1d5a-4c4f-890c-df79ae81093c None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:20.246882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:20.307516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd0fed-b09f-45d5-a45b-66c6949282e6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:20.341701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3700f69-1d5a-4c4f-890c-df79ae81093c None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:20.377584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181f7ce-33ec-4315-8897-24ffc8b1df58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:20.379284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181f7ce-33ec-4315-8897-24ffc8b1df58 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:20.379990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181f7ce-33ec-4315-8897-24ffc8b1df58 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:20.380669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181f7ce-33ec-4315-8897-24ffc8b1df58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:20.393734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068dabf-7c02-4d51-b63c-9017a44b07fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:20.394699 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068dabf-7c02-4d51-b63c-9017a44b07fe None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:20.395137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068dabf-7c02-4d51-b63c-9017a44b07fe None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:20.395586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068dabf-7c02-4d51-b63c-9017a44b07fe None admin] PATH_INFO: `/v3/users/glance` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:20.403291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181f7ce-33ec-4315-8897-24ffc8b1df58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:20.406482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181f7ce-33ec-4315-8897-24ffc8b1df58 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:20.407050 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068dabf-7c02-4d51-b63c-9017a44b07fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:20.408773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068dabf-7c02-4d51-b63c-9017a44b07fe None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-1068dabf-7c02-4d51-b63c-9017a44b07fe None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:20.427589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:20.429697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:20.431489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:20.431489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:20.431489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:20.431489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:20.431489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:20.431489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:20.431489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:20.431489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:20.431489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 08:22:20.431489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:20.431489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 330/657] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:20.431489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2b91a-8182-4288-bcec-4277a41ef8b0 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:20.432762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2b91a-8182-4288-bcec-4277a41ef8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:20.432762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2b91a-8182-4288-bcec-4277a41ef8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:20.447792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-2181f7ce-33ec-4315-8897-24ffc8b1df58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:20.452586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:20.454599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:20.457137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:20.457137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:20.457137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:20.457137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:20.457137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 328/658] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:20.457137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181f7ce-33ec-4315-8897-24ffc8b1df58 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:20.462125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-84d2b91a-8182-4288-bcec-4277a41ef8b0 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:20.463885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-84d2b91a-8182-4288-bcec-4277a41ef8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:20.556346 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 331/659] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:20.557405 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2b91a-8182-4288-bcec-4277a41ef8b0 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:20.658359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181f7ce-33ec-4315-8897-24ffc8b1df58 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:20.689716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:20.704951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2b91a-8182-4288-bcec-4277a41ef8b0 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:20.708673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f9e5b-212b-400a-b27f-c3cbbd3aa49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:20.710143 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f9e5b-212b-400a-b27f-c3cbbd3aa49b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:20.710709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f9e5b-212b-400a-b27f-c3cbbd3aa49b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:20.711252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f9e5b-212b-400a-b27f-c3cbbd3aa49b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:20.725305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f9e5b-212b-400a-b27f-c3cbbd3aa49b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:20.727930 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f9e5b-212b-400a-b27f-c3cbbd3aa49b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:20.743091 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107a6e0-6765-48eb-a169-054245a0318f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:20.743769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107a6e0-6765-48eb-a169-054245a0318f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:20.744055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107a6e0-6765-48eb-a169-054245a0318f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:20.744332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107a6e0-6765-48eb-a169-054245a0318f None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:20.745218 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107a6e0-6765-48eb-a169-054245a0318f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:20.745530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107a6e0-6765-48eb-a169-054245a0318f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:20.748056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107a6e0-6765-48eb-a169-054245a0318f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:20.751119 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-397f9e5b-212b-400a-b27f-c3cbbd3aa49b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:20.755753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:20.756733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:20.757758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:20.757758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:20.757758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:20.757758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:20.757758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:20.757758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:20.757758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:20.757758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:20.757758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 08:22:20.757758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:20.757758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 329/660] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:20.758948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f9e5b-212b-400a-b27f-c3cbbd3aa49b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:20.774149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 332/661] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Jun 7 08:22:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:20.775124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107a6e0-6765-48eb-a169-054245a0318f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:20.934966 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f9e5b-212b-400a-b27f-c3cbbd3aa49b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:20.953416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107a6e0-6765-48eb-a169-054245a0318f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:20.983064 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ec513-b6d8-4251-80fe-41ab595c923a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:20.983866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ec513-b6d8-4251-80fe-41ab595c923a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:20.984147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ec513-b6d8-4251-80fe-41ab595c923a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:20.984418 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ec513-b6d8-4251-80fe-41ab595c923a None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:20.985328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ec513-b6d8-4251-80fe-41ab595c923a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:20.985635 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ec513-b6d8-4251-80fe-41ab595c923a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:20.987042 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ec513-b6d8-4251-80fe-41ab595c923a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:20.998846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12ab80-c728-48d9-90c5-6a8c9aa3aca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:20.999671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12ab80-c728-48d9-90c5-6a8c9aa3aca0 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:20.999959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12ab80-c728-48d9-90c5-6a8c9aa3aca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:21.000235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12ab80-c728-48d9-90c5-6a8c9aa3aca0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:21.009571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 330/662] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:21.011347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ec513-b6d8-4251-80fe-41ab595c923a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:21.020469 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12ab80-c728-48d9-90c5-6a8c9aa3aca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-fc12ab80-c728-48d9-90c5-6a8c9aa3aca0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:21.034680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:21.036854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 333/663] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:21.038658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12ab80-c728-48d9-90c5-6a8c9aa3aca0 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:21.097838 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:21.145916 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:21.206106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ec513-b6d8-4251-80fe-41ab595c923a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:21.227598 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12ab80-c728-48d9-90c5-6a8c9aa3aca0 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:21.278660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf47d0-ae7b-4be5-a35d-6a264192d8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:21.280009 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf47d0-ae7b-4be5-a35d-6a264192d8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:21.280466 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf47d0-ae7b-4be5-a35d-6a264192d8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:21.280923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf47d0-ae7b-4be5-a35d-6a264192d8a0 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:21.282557 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf47d0-ae7b-4be5-a35d-6a264192d8a0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:21.283100 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf47d0-ae7b-4be5-a35d-6a264192d8a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:21.285851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf47d0-ae7b-4be5-a35d-6a264192d8a0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:21.305673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ca688-260e-4011-9786-b46465899b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:21.306809 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ca688-260e-4011-9786-b46465899b4b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:21.307321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ca688-260e-4011-9786-b46465899b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:21.307837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ca688-260e-4011-9786-b46465899b4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:21.326912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 331/664] 10.222.0.39 () {56 vars in 1139 bytes} [Tue Jun 7 08:22:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:21.326912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf47d0-ae7b-4be5-a35d-6a264192d8a0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:21.470683 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ca688-260e-4011-9786-b46465899b4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:21.473927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ca688-260e-4011-9786-b46465899b4b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:21.496473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:21.503501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf47d0-ae7b-4be5-a35d-6a264192d8a0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:21.553789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1851ee7-426b-4875-9f28-3865805652ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:21.554588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1851ee7-426b-4875-9f28-3865805652ea None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:21.554870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1851ee7-426b-4875-9f28-3865805652ea None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:21.555143 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1851ee7-426b-4875-9f28-3865805652ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:21.564062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1851ee7-426b-4875-9f28-3865805652ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:21.565682 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1851ee7-426b-4875-9f28-3865805652ea None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:21.592597 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-b1851ee7-426b-4875-9f28-3865805652ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:21.592865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:21.595472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:21.597469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:21.597469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:21.597469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:21.597469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:21.597469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:21.597469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:21.597469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:21.597469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:21.597469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 332/665] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:21.597469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1851ee7-426b-4875-9f28-3865805652ea None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:21.668748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 334/666] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:22:21.669942 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ca688-260e-4011-9786-b46465899b4b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:21.802118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1851ee7-426b-4875-9f28-3865805652ea None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:21.821511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ca688-260e-4011-9786-b46465899b4b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:21.874700 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a55d31-14e3-4559-8aa1-96664ee322ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:21.875826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a55d31-14e3-4559-8aa1-96664ee322ed None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:21.876304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a55d31-14e3-4559-8aa1-96664ee322ed None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:21.876756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a55d31-14e3-4559-8aa1-96664ee322ed None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:21.878139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a55d31-14e3-4559-8aa1-96664ee322ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:21.878695 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a55d31-14e3-4559-8aa1-96664ee322ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:21.881347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a55d31-14e3-4559-8aa1-96664ee322ed None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:21.885170 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083e624-50eb-437a-bd50-74fb55735657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:21.886059 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083e624-50eb-437a-bd50-74fb55735657 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:21.886524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083e624-50eb-437a-bd50-74fb55735657 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:21.889410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083e624-50eb-437a-bd50-74fb55735657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:21.898587 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 333/667] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:21.900167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a55d31-14e3-4559-8aa1-96664ee322ed None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:21.925771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:21.942179 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:22.012918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083e624-50eb-437a-bd50-74fb55735657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:22.015529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083e624-50eb-437a-bd50-74fb55735657 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:22.117315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a55d31-14e3-4559-8aa1-96664ee322ed None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:22.187983 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b5959-7afb-419d-b348-856b2e8b5ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:22.188660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b5959-7afb-419d-b348-856b2e8b5ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:22.188943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b5959-7afb-419d-b348-856b2e8b5ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:22.189251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b5959-7afb-419d-b348-856b2e8b5ae2 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:22.190053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b5959-7afb-419d-b348-856b2e8b5ae2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:22.190361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b5959-7afb-419d-b348-856b2e8b5ae2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:22.191825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b5959-7afb-419d-b348-856b2e8b5ae2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:22.205834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 335/668] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:22:22.205834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083e624-50eb-437a-bd50-74fb55735657 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:22.218783 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 334/669] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:22.220184 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b5959-7afb-419d-b348-856b2e8b5ae2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:22.324588 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:22.392773 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:22.412166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083e624-50eb-437a-bd50-74fb55735657 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:22.458447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b5959-7afb-419d-b348-856b2e8b5ae2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:22.489741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94897181-cdc1-456a-abf1-cd0a3a46b3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:22.491550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94897181-cdc1-456a-abf1-cd0a3a46b3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:22.491995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94897181-cdc1-456a-abf1-cd0a3a46b3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:22.492327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94897181-cdc1-456a-abf1-cd0a3a46b3a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:22.507481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94897181-cdc1-456a-abf1-cd0a3a46b3a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:22.510131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94897181-cdc1-456a-abf1-cd0a3a46b3a6 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:22.514429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea409495-2ed8-492d-a4a2-658c2a7a0190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:22.515521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea409495-2ed8-492d-a4a2-658c2a7a0190 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:22.515981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea409495-2ed8-492d-a4a2-658c2a7a0190 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:22.516436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea409495-2ed8-492d-a4a2-658c2a7a0190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:22.534975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea409495-2ed8-492d-a4a2-658c2a7a0190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:22.537526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea409495-2ed8-492d-a4a2-658c2a7a0190 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-94897181-cdc1-456a-abf1-cd0a3a46b3a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:22.542451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:22.544896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:22.547799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:22.547799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:22.547799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:22.547799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:22.547799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 336/670] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:22.547799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94897181-cdc1-456a-abf1-cd0a3a46b3a6 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-ea409495-2ed8-492d-a4a2-658c2a7a0190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:22.593311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:22.601699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:22.604862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:22.604862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:22.604862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:22.604862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:22.604862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 335/671] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:22.604862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea409495-2ed8-492d-a4a2-658c2a7a0190 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:22.697754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94897181-cdc1-456a-abf1-cd0a3a46b3a6 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:22.707489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:22.760203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaba8b0-5bf8-47db-9e54-836d1765a0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:22.761261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaba8b0-5bf8-47db-9e54-836d1765a0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:22.762318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaba8b0-5bf8-47db-9e54-836d1765a0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:22.762743 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaba8b0-5bf8-47db-9e54-836d1765a0c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:22.788660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaba8b0-5bf8-47db-9e54-836d1765a0c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:22.791961 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaba8b0-5bf8-47db-9e54-836d1765a0c4 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:22.806014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea409495-2ed8-492d-a4a2-658c2a7a0190 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-6aaba8b0-5bf8-47db-9e54-836d1765a0c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:22.833157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:22.835426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:22.838233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:22.838233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:22.838233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:22.838233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:22.838233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 337/672] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:22.841458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaba8b0-5bf8-47db-9e54-836d1765a0c4 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:22.866909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:22.876308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466320fe-14da-41c3-8f95-a4d773fa0b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:22.877438 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466320fe-14da-41c3-8f95-a4d773fa0b1f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:22.877922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466320fe-14da-41c3-8f95-a4d773fa0b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:22.878406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466320fe-14da-41c3-8f95-a4d773fa0b1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:22.880054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466320fe-14da-41c3-8f95-a4d773fa0b1f None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=2b6943045d594d729fc7c4b15cbaa07e) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:22.880620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466320fe-14da-41c3-8f95-a4d773fa0b1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:22.883314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466320fe-14da-41c3-8f95-a4d773fa0b1f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:22.894574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 336/673] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:22 2022] GET /identity/v3/role_assignments?user.id=2b6943045d594d729fc7c4b15cbaa07e&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:22.896465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466320fe-14da-41c3-8f95-a4d773fa0b1f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:23.046734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaba8b0-5bf8-47db-9e54-836d1765a0c4 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:23.084777 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466320fe-14da-41c3-8f95-a4d773fa0b1f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:23.097799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c586f8-e1f6-4c8b-857a-0f1abd66ac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:23.098508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c586f8-e1f6-4c8b-857a-0f1abd66ac7e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:23.098790 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c586f8-e1f6-4c8b-857a-0f1abd66ac7e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:23.099064 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c586f8-e1f6-4c8b-857a-0f1abd66ac7e None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:23.099959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c586f8-e1f6-4c8b-857a-0f1abd66ac7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:23.100274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c586f8-e1f6-4c8b-857a-0f1abd66ac7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:23.101781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c586f8-e1f6-4c8b-857a-0f1abd66ac7e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:23.128428 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 338/674] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:23.128428 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c586f8-e1f6-4c8b-857a-0f1abd66ac7e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:23.161666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:23.171370 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fbbcd-da64-4463-af02-0b910b7ceaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:23.172574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fbbcd-da64-4463-af02-0b910b7ceaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:23.173152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fbbcd-da64-4463-af02-0b910b7ceaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:23.173621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fbbcd-da64-4463-af02-0b910b7ceaa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:23.174943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fbbcd-da64-4463-af02-0b910b7ceaa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:23.175481 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fbbcd-da64-4463-af02-0b910b7ceaa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:23.178049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fbbcd-da64-4463-af02-0b910b7ceaa1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:23.207179 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 337/675] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:23.212138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb4941-64a9-4337-b811-c41ee8a70587 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:23.212138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb4941-64a9-4337-b811-c41ee8a70587 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:23.212138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb4941-64a9-4337-b811-c41ee8a70587 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:23.213614 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 338/676] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:23.214554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb4941-64a9-4337-b811-c41ee8a70587 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:23.300923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:23.307521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c586f8-e1f6-4c8b-857a-0f1abd66ac7e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:23.354942 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb240d4-12ba-49fa-b95e-40016ac2065f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:23.356577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb240d4-12ba-49fa-b95e-40016ac2065f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:23.357027 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb240d4-12ba-49fa-b95e-40016ac2065f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:23.357517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb240d4-12ba-49fa-b95e-40016ac2065f None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:23.359010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb240d4-12ba-49fa-b95e-40016ac2065f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:23.359606 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb240d4-12ba-49fa-b95e-40016ac2065f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:23.362162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb240d4-12ba-49fa-b95e-40016ac2065f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:23.369437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb4941-64a9-4337-b811-c41ee8a70587 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:23.388110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 339/677] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:23.388110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb240d4-12ba-49fa-b95e-40016ac2065f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:23.432703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed6a02-34a7-49c7-bff2-2d4773486bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:23.433799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed6a02-34a7-49c7-bff2-2d4773486bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:23.434219 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed6a02-34a7-49c7-bff2-2d4773486bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:23.434691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed6a02-34a7-49c7-bff2-2d4773486bb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:23.445936 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed6a02-34a7-49c7-bff2-2d4773486bb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:23.448463 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed6a02-34a7-49c7-bff2-2d4773486bb1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-e0ed6a02-34a7-49c7-bff2-2d4773486bb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:23.474879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:23.476817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:23.478826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:23.478826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:23.478826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:23.478826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:23.478826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:23.478826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:23.478826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:23.478826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:23.478826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 339/678] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:23.479847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566980e-d66a-4a83-b988-91b0ca12b3a9 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:23.480253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566980e-d66a-4a83-b988-91b0ca12b3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:23.481362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566980e-d66a-4a83-b988-91b0ca12b3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:23.504768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5566980e-d66a-4a83-b988-91b0ca12b3a9 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:23.506143 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5566980e-d66a-4a83-b988-91b0ca12b3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:23.522358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:23.566192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb240d4-12ba-49fa-b95e-40016ac2065f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:23.624692 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561ee84-7a3f-418e-be8f-bad3e93ab621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:23.625833 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561ee84-7a3f-418e-be8f-bad3e93ab621 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:23.626307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561ee84-7a3f-418e-be8f-bad3e93ab621 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:23.626812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561ee84-7a3f-418e-be8f-bad3e93ab621 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:23.633393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 340/679] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:23.635245 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566980e-d66a-4a83-b988-91b0ca12b3a9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:23.642398 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561ee84-7a3f-418e-be8f-bad3e93ab621 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:23.644070 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561ee84-7a3f-418e-be8f-bad3e93ab621 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-a561ee84-7a3f-418e-be8f-bad3e93ab621 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:23.675449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:23.676686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:23.678212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:23.678212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:23.678212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:23.678212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:23.678212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:23.678212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:23.678212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:23.678212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:23.678212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:22:23.678212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:23.678212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 340/680] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:23.679276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561ee84-7a3f-418e-be8f-bad3e93ab621 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:23.715390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:23.792329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566980e-d66a-4a83-b988-91b0ca12b3a9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:23.811145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561ee84-7a3f-418e-be8f-bad3e93ab621 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:23.854738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03ce07-1797-432f-b54f-1140c64ad861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:23.856221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03ce07-1797-432f-b54f-1140c64ad861 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:23.856614 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03ce07-1797-432f-b54f-1140c64ad861 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:23.865425 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03ce07-1797-432f-b54f-1140c64ad861 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:23.875406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03ce07-1797-432f-b54f-1140c64ad861 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:23.876870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03ce07-1797-432f-b54f-1140c64ad861 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:23.878892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12d82b-0bb1-4f4e-bcc4-f7f0c2dae561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:23.880089 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12d82b-0bb1-4f4e-bcc4-f7f0c2dae561 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:23.880608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12d82b-0bb1-4f4e-bcc4-f7f0c2dae561 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:23.881149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12d82b-0bb1-4f4e-bcc4-f7f0c2dae561 None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:23.882792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12d82b-0bb1-4f4e-bcc4-f7f0c2dae561 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:23.883352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12d82b-0bb1-4f4e-bcc4-f7f0c2dae561 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:23.886392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12d82b-0bb1-4f4e-bcc4-f7f0c2dae561 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-8b03ce07-1797-432f-b54f-1140c64ad861 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:23.898572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:23.899511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:23.900982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:23.900982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:23.900982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:23.900982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:23.900982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:23.900982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:23.900982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:23.900982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:23.903255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:23.905243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 341/681] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:23.906348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03ce07-1797-432f-b54f-1140c64ad861 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:23.914312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 341/682] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:23.917461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12d82b-0bb1-4f4e-bcc4-f7f0c2dae561 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:24.078474 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12d82b-0bb1-4f4e-bcc4-f7f0c2dae561 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:24.086327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03ce07-1797-432f-b54f-1140c64ad861 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:24.102891 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:24.131319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb47d0-9a40-449f-afd9-483c6047625e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:24.132384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb47d0-9a40-449f-afd9-483c6047625e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:24.132879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb47d0-9a40-449f-afd9-483c6047625e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:24.133551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb47d0-9a40-449f-afd9-483c6047625e None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:24.135372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb47d0-9a40-449f-afd9-483c6047625e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:24.135900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb47d0-9a40-449f-afd9-483c6047625e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:24.139111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb47d0-9a40-449f-afd9-483c6047625e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:24.144075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21597a5e-25ba-46a3-911e-7b684faaf4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:24.145288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21597a5e-25ba-46a3-911e-7b684faaf4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:24.145740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21597a5e-25ba-46a3-911e-7b684faaf4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:24.146197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21597a5e-25ba-46a3-911e-7b684faaf4b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:24.159931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21597a5e-25ba-46a3-911e-7b684faaf4b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-21597a5e-25ba-46a3-911e-7b684faaf4b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:24.169139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:24.171045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 342/683] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:24.173982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21597a5e-25ba-46a3-911e-7b684faaf4b6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:24.186675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 342/684] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Jun 7 08:22:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:24.188581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb47d0-9a40-449f-afd9-483c6047625e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:24.289689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:24.360199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21597a5e-25ba-46a3-911e-7b684faaf4b6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:24.374200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb47d0-9a40-449f-afd9-483c6047625e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:24.424486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcdd34-1d38-4ebf-8012-767a8e2e12eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:24.425532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcdd34-1d38-4ebf-8012-767a8e2e12eb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:24.425982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcdd34-1d38-4ebf-8012-767a8e2e12eb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:24.426454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcdd34-1d38-4ebf-8012-767a8e2e12eb None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:24.427806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcdd34-1d38-4ebf-8012-767a8e2e12eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:24.428319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcdd34-1d38-4ebf-8012-767a8e2e12eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:24.430497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcdd34-1d38-4ebf-8012-767a8e2e12eb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:24.432047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e653ec-e042-4373-b689-ef4e515de5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:24.433225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e653ec-e042-4373-b689-ef4e515de5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:24.433729 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e653ec-e042-4373-b689-ef4e515de5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:24.434189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e653ec-e042-4373-b689-ef4e515de5a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:24.435778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e653ec-e042-4373-b689-ef4e515de5a2 None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=d1e4531ac696445ba5bfa9870984befd) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:24.436339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e653ec-e042-4373-b689-ef4e515de5a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:24.439061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e653ec-e042-4373-b689-ef4e515de5a2 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:24.456528 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 343/685] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:24.456528 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcdd34-1d38-4ebf-8012-767a8e2e12eb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:24.460957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 343/686] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:24 2022] GET /identity/v3/role_assignments?user.id=d1e4531ac696445ba5bfa9870984befd&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:24.462474 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e653ec-e042-4373-b689-ef4e515de5a2 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:24.509440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:24.621964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e653ec-e042-4373-b689-ef4e515de5a2 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:24.652354 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:24.663469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcdd34-1d38-4ebf-8012-767a8e2e12eb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:24.674139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20ff8c-e1af-4e6b-ad63-cb4b73cce7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:24.674918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20ff8c-e1af-4e6b-ad63-cb4b73cce7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:24.675209 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20ff8c-e1af-4e6b-ad63-cb4b73cce7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:24.675538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20ff8c-e1af-4e6b-ad63-cb4b73cce7d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:24.735941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3453e4-f917-42d2-b042-4c9685c5f382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:24.737730 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3453e4-f917-42d2-b042-4c9685c5f382 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:24.738598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3453e4-f917-42d2-b042-4c9685c5f382 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:24.739384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3453e4-f917-42d2-b042-4c9685c5f382 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:24.741163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3453e4-f917-42d2-b042-4c9685c5f382 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:24.742041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3453e4-f917-42d2-b042-4c9685c5f382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:24.744806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3453e4-f917-42d2-b042-4c9685c5f382 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:24.761746 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 344/687] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:24.763755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3453e4-f917-42d2-b042-4c9685c5f382 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:24.783402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20ff8c-e1af-4e6b-ad63-cb4b73cce7d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:24.783402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20ff8c-e1af-4e6b-ad63-cb4b73cce7d2 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:24.962988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 344/688] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:22:24.962988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20ff8c-e1af-4e6b-ad63-cb4b73cce7d2 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:24.964274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:24.989792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3453e4-f917-42d2-b042-4c9685c5f382 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:25.005352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:25.053291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2130578-17a6-4f86-85c1-34c42ff8efc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:25.054475 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2130578-17a6-4f86-85c1-34c42ff8efc7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:25.054862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2130578-17a6-4f86-85c1-34c42ff8efc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:25.055338 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2130578-17a6-4f86-85c1-34c42ff8efc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:25.056904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2130578-17a6-4f86-85c1-34c42ff8efc7 None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=7ddf89deb28749eda2d26ad66bc282f8) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:25.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2130578-17a6-4f86-85c1-34c42ff8efc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:25.060658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2130578-17a6-4f86-85c1-34c42ff8efc7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:25.077134 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 345/689] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:24 2022] GET /identity/v3/role_assignments?user.id=7ddf89deb28749eda2d26ad66bc282f8&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:25.077134 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2130578-17a6-4f86-85c1-34c42ff8efc7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:25.147318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20ff8c-e1af-4e6b-ad63-cb4b73cce7d2 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:25.191187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49423f90-5e59-428f-85d9-731107c2d281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:25.192481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49423f90-5e59-428f-85d9-731107c2d281 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:25.192965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49423f90-5e59-428f-85d9-731107c2d281 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:25.193466 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49423f90-5e59-428f-85d9-731107c2d281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:25.207350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49423f90-5e59-428f-85d9-731107c2d281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:25.209324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49423f90-5e59-428f-85d9-731107c2d281 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-49423f90-5e59-428f-85d9-731107c2d281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:25.234026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:25.234968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:25.235862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:25.235862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:25.235862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:25.235862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:25.238587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 345/690] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:25.247415 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49423f90-5e59-428f-85d9-731107c2d281 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:25.342337 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2130578-17a6-4f86-85c1-34c42ff8efc7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:25.414212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f691f55-d2d5-49ad-b51b-38a766c64ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:25.420280 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f691f55-d2d5-49ad-b51b-38a766c64ede None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:25.420920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f691f55-d2d5-49ad-b51b-38a766c64ede None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:25.421446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f691f55-d2d5-49ad-b51b-38a766c64ede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:25.448456 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f691f55-d2d5-49ad-b51b-38a766c64ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:25.451360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f691f55-d2d5-49ad-b51b-38a766c64ede None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:25.463803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:25.509406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49423f90-5e59-428f-85d9-731107c2d281 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-7f691f55-d2d5-49ad-b51b-38a766c64ede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:25.517852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:25.518865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:25.519833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:25.519833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:25.519833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:25.519833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:25.525006 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 346/691] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:25.531327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f691f55-d2d5-49ad-b51b-38a766c64ede None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:25.536370 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:25.593717 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bda3a-0fe4-4431-897a-1fd457cb0072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:25.594792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bda3a-0fe4-4431-897a-1fd457cb0072 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:25.595308 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bda3a-0fe4-4431-897a-1fd457cb0072 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:25.596055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bda3a-0fe4-4431-897a-1fd457cb0072 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:25.597019 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bda3a-0fe4-4431-897a-1fd457cb0072 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:25.597595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bda3a-0fe4-4431-897a-1fd457cb0072 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:25.599807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bda3a-0fe4-4431-897a-1fd457cb0072 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:25.619247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 346/692] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:25.629558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bda3a-0fe4-4431-897a-1fd457cb0072 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:25.710432 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f691f55-d2d5-49ad-b51b-38a766c64ede None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:25.754978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fa431-d80f-4dbe-8d1c-0288c7ff0ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:25.756046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fa431-d80f-4dbe-8d1c-0288c7ff0ded None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:25.756543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fa431-d80f-4dbe-8d1c-0288c7ff0ded None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:25.757016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fa431-d80f-4dbe-8d1c-0288c7ff0ded None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:25.757956 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fa431-d80f-4dbe-8d1c-0288c7ff0ded None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:25.758513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fa431-d80f-4dbe-8d1c-0288c7ff0ded None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:25.760210 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fa431-d80f-4dbe-8d1c-0288c7ff0ded None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:25.780793 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 347/693] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:25.782510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fa431-d80f-4dbe-8d1c-0288c7ff0ded None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:25.843739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bda3a-0fe4-4431-897a-1fd457cb0072 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:25.906013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:25.908105 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:25.917912 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85d04e-6982-4a0e-a0af-f9a50e6d0ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:25.919088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85d04e-6982-4a0e-a0af-f9a50e6d0ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:25.919383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85d04e-6982-4a0e-a0af-f9a50e6d0ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:25.919634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85d04e-6982-4a0e-a0af-f9a50e6d0ba1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:25.936575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85d04e-6982-4a0e-a0af-f9a50e6d0ba1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:25.938079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85d04e-6982-4a0e-a0af-f9a50e6d0ba1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-1b85d04e-6982-4a0e-a0af-f9a50e6d0ba1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:25.974587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:25.977360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:25.979323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:25.979323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:25.979323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:25.979323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:25.979323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:25.979323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:25.979323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:25.979323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:25.979323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:22:25.979323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:25.979323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 347/694] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jun 7 08:22:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:25.982769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fa431-d80f-4dbe-8d1c-0288c7ff0ded None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:25.986910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85d04e-6982-4a0e-a0af-f9a50e6d0ba1 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:26.048752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ab34b-adf3-4c5d-a23e-3aa967160839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:26.049519 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ab34b-adf3-4c5d-a23e-3aa967160839 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:26.049789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ab34b-adf3-4c5d-a23e-3aa967160839 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:26.050068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ab34b-adf3-4c5d-a23e-3aa967160839 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:26.057620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ab34b-adf3-4c5d-a23e-3aa967160839 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:26.060191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ab34b-adf3-4c5d-a23e-3aa967160839 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-152ab34b-adf3-4c5d-a23e-3aa967160839 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:26.076096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:26.078140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:26.080305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:26.080305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:26.080305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:26.080305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:26.080305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:26.080305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:26.080305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:26.080305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:26.084046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 348/695] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:26.086294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ab34b-adf3-4c5d-a23e-3aa967160839 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:26.241799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85d04e-6982-4a0e-a0af-f9a50e6d0ba1 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:26.310587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec739f-5cc0-4f61-ac2e-b6a9099cce43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:26.311170 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec739f-5cc0-4f61-ac2e-b6a9099cce43 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:26.311452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec739f-5cc0-4f61-ac2e-b6a9099cce43 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:26.311723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec739f-5cc0-4f61-ac2e-b6a9099cce43 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:26.312579 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec739f-5cc0-4f61-ac2e-b6a9099cce43 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:26.312886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec739f-5cc0-4f61-ac2e-b6a9099cce43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:26.319728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec739f-5cc0-4f61-ac2e-b6a9099cce43 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:26.336748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ab34b-adf3-4c5d-a23e-3aa967160839 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:26.374373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 348/696] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jun 7 08:22:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:26.379081 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e1757-afbd-49be-aeec-5861a7b03681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:26.379870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e1757-afbd-49be-aeec-5861a7b03681 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:26.380194 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e1757-afbd-49be-aeec-5861a7b03681 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:26.380351 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec739f-5cc0-4f61-ac2e-b6a9099cce43 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:26.380490 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e1757-afbd-49be-aeec-5861a7b03681 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:26.381281 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e1757-afbd-49be-aeec-5861a7b03681 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:26.381600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e1757-afbd-49be-aeec-5861a7b03681 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:26.382421 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:26.386288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e1757-afbd-49be-aeec-5861a7b03681 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:26.418937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 349/697] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:26.421224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e1757-afbd-49be-aeec-5861a7b03681 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:26.515107 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:26.591504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec739f-5cc0-4f61-ac2e-b6a9099cce43 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:26.658950 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e1757-afbd-49be-aeec-5861a7b03681 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:26.670525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69189ba-0698-4a2b-b2c9-589b79dcacb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:26.671202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69189ba-0698-4a2b-b2c9-589b79dcacb5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:26.671481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69189ba-0698-4a2b-b2c9-589b79dcacb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:26.671768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69189ba-0698-4a2b-b2c9-589b79dcacb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:26.734608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa95c1-8450-4a09-9d4d-6f2b278d01c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:26.735486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa95c1-8450-4a09-9d4d-6f2b278d01c8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:26.735669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa95c1-8450-4a09-9d4d-6f2b278d01c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:26.735800 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa95c1-8450-4a09-9d4d-6f2b278d01c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:26.736648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa95c1-8450-4a09-9d4d-6f2b278d01c8 None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=586ddffa77a54521b42b2aa50f16cb92) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:26.737078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa95c1-8450-4a09-9d4d-6f2b278d01c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:26.738632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa95c1-8450-4a09-9d4d-6f2b278d01c8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:26.754387 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 350/698] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:26 2022] GET /identity/v3/role_assignments?user.id=586ddffa77a54521b42b2aa50f16cb92&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 608 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:26.765602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69189ba-0698-4a2b-b2c9-589b79dcacb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:26.767124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69189ba-0698-4a2b-b2c9-589b79dcacb5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:26.772281 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c336ed-6d63-4883-bb51-30f2e4770d41 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:26.773337 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c336ed-6d63-4883-bb51-30f2e4770d41 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:26.774169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c336ed-6d63-4883-bb51-30f2e4770d41 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:26.777804 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 351/699] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:26.788305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109b985-5fec-4458-8a96-fdd5b4e4bdb3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:26.789426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109b985-5fec-4458-8a96-fdd5b4e4bdb3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:26.790398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109b985-5fec-4458-8a96-fdd5b4e4bdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:26.826938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f109b985-5fec-4458-8a96-fdd5b4e4bdb3 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:26.828502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f109b985-5fec-4458-8a96-fdd5b4e4bdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:26.882356 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:26.883109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:26.912931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 352/700] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:22:26.955311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109b985-5fec-4458-8a96-fdd5b4e4bdb3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:26.991054 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 349/701] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:22:26.999368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69189ba-0698-4a2b-b2c9-589b79dcacb5 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:27.149814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109b985-5fec-4458-8a96-fdd5b4e4bdb3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:27.173259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69189ba-0698-4a2b-b2c9-589b79dcacb5 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:27.196966 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f873-62a0-487b-9596-034db5db0dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:27.197873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f873-62a0-487b-9596-034db5db0dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:27.198157 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f873-62a0-487b-9596-034db5db0dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:27.198427 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f873-62a0-487b-9596-034db5db0dd6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:27.199720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4f873-62a0-487b-9596-034db5db0dd6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:27.201913 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4f873-62a0-487b-9596-034db5db0dd6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-70f4f873-62a0-487b-9596-034db5db0dd6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:27.220269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:27.222285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:27.223942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 353/702] 10.222.0.39 () {56 vars in 1129 bytes} [Tue Jun 7 08:22:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:27.225643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dae75-50fd-42f4-8177-480d95bf86dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:27.225643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dae75-50fd-42f4-8177-480d95bf86dd None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:27.225643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dae75-50fd-42f4-8177-480d95bf86dd None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:27.225643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dae75-50fd-42f4-8177-480d95bf86dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:27.230057 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f873-62a0-487b-9596-034db5db0dd6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:27.252278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6dae75-50fd-42f4-8177-480d95bf86dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:27.255115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6dae75-50fd-42f4-8177-480d95bf86dd None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:27.282896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-ca6dae75-50fd-42f4-8177-480d95bf86dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:27.293467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:27.295161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:27.296902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:27.296902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:27.296902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:27.296902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:27.296902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 350/703] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:27.303456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dae75-50fd-42f4-8177-480d95bf86dd None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:27.307218 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:27.373189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f873-62a0-487b-9596-034db5db0dd6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:27.454202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d956a1-c2fe-494f-a00b-d6dfd720a9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:27.455221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d956a1-c2fe-494f-a00b-d6dfd720a9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:27.455710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d956a1-c2fe-494f-a00b-d6dfd720a9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:27.456191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d956a1-c2fe-494f-a00b-d6dfd720a9e0 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:27.457498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d956a1-c2fe-494f-a00b-d6dfd720a9e0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:27.458011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d956a1-c2fe-494f-a00b-d6dfd720a9e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:27.460512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d956a1-c2fe-494f-a00b-d6dfd720a9e0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:27.477369 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 354/704] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Jun 7 08:22:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:27.477712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dae75-50fd-42f4-8177-480d95bf86dd None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:27.480578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d956a1-c2fe-494f-a00b-d6dfd720a9e0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:27.561421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2402464-0d07-4610-a1d5-7c81638bc79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:27.562556 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2402464-0d07-4610-a1d5-7c81638bc79c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:27.563047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2402464-0d07-4610-a1d5-7c81638bc79c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:27.563500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2402464-0d07-4610-a1d5-7c81638bc79c None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:27.565310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2402464-0d07-4610-a1d5-7c81638bc79c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:27.565848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2402464-0d07-4610-a1d5-7c81638bc79c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:27.568831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2402464-0d07-4610-a1d5-7c81638bc79c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:27.596885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 351/705] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:27.604196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2402464-0d07-4610-a1d5-7c81638bc79c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:27.690216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d956a1-c2fe-494f-a00b-d6dfd720a9e0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:27.708669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:27.749897 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:27.783850 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb831d-6f14-499d-b41e-d287e5037b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:27.784592 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb831d-6f14-499d-b41e-d287e5037b21 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:27.784873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb831d-6f14-499d-b41e-d287e5037b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:27.785137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb831d-6f14-499d-b41e-d287e5037b21 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:27.785982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb831d-6f14-499d-b41e-d287e5037b21 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:27.786293 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb831d-6f14-499d-b41e-d287e5037b21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:27.787856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb831d-6f14-499d-b41e-d287e5037b21 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:27.802223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 355/706] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Jun 7 08:22:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:27.827988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2402464-0d07-4610-a1d5-7c81638bc79c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:27.880786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1262223-aae3-4319-bc28-9671481aedc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:27.881858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1262223-aae3-4319-bc28-9671481aedc7 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:27.882150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1262223-aae3-4319-bc28-9671481aedc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:27.882484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1262223-aae3-4319-bc28-9671481aedc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:27.903261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1262223-aae3-4319-bc28-9671481aedc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:27.904973 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1262223-aae3-4319-bc28-9671481aedc7 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-e1262223-aae3-4319-bc28-9671481aedc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:27.925617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:27.927777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:27.930298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:27.930298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:27.930298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:27.930298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:27.930298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:27.930298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:27.930298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:27.930298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:27.937520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 352/707] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:27.941439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb831d-6f14-499d-b41e-d287e5037b21 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:28.082860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0798ecf-3762-4eb0-ac1d-d0e76b318c6b None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:28.083661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0798ecf-3762-4eb0-ac1d-d0e76b318c6b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:28.083661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0798ecf-3762-4eb0-ac1d-d0e76b318c6b None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:28.086998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 353/708] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:28.093719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ce76a-4bc9-4adf-8450-023b1dcb9109 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:28.094047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ce76a-4bc9-4adf-8450-023b1dcb9109 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:28.094264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ce76a-4bc9-4adf-8450-023b1dcb9109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:28.113654 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb831d-6f14-499d-b41e-d287e5037b21 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:28.124694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-ad2ce76a-4bc9-4adf-8450-023b1dcb9109 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:28.125351 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ad2ce76a-4bc9-4adf-8450-023b1dcb9109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:28.176290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375278b-4c94-4eb6-a9f3-129564958bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:28.177033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375278b-4c94-4eb6-a9f3-129564958bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:28.177349 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375278b-4c94-4eb6-a9f3-129564958bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:28.177628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375278b-4c94-4eb6-a9f3-129564958bf2 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:28.178482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375278b-4c94-4eb6-a9f3-129564958bf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:28.178794 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375278b-4c94-4eb6-a9f3-129564958bf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:28.180235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375278b-4c94-4eb6-a9f3-129564958bf2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:28.197197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 356/709] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:28.202274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375278b-4c94-4eb6-a9f3-129564958bf2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:28.238531 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:28.271342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 354/710] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:28.305374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ce76a-4bc9-4adf-8450-023b1dcb9109 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:28.377753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:28.393213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375278b-4c94-4eb6-a9f3-129564958bf2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:28.457220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ce76a-4bc9-4adf-8450-023b1dcb9109 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:28.496282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8f81b-bd68-4cce-afad-b9031d126015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:28.497630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8f81b-bd68-4cce-afad-b9031d126015 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:28.497948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8f81b-bd68-4cce-afad-b9031d126015 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:28.498205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8f81b-bd68-4cce-afad-b9031d126015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:28.499279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f91ba-f2c2-4aeb-820d-0592cbe52999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:28.501586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f91ba-f2c2-4aeb-820d-0592cbe52999 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:28.502438 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f91ba-f2c2-4aeb-820d-0592cbe52999 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:28.503217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f91ba-f2c2-4aeb-820d-0592cbe52999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:28.505313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f91ba-f2c2-4aeb-820d-0592cbe52999 None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=3f2a2bf913d1412f996acf4401ba7673) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:28.506322 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f91ba-f2c2-4aeb-820d-0592cbe52999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:28.513136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8f81b-bd68-4cce-afad-b9031d126015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:28.513405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f91ba-f2c2-4aeb-820d-0592cbe52999 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:28.513553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8f81b-bd68-4cce-afad-b9031d126015 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:28.526019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 357/711] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:28 2022] GET /identity/v3/role_assignments?user.id=3f2a2bf913d1412f996acf4401ba7673&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-bcf8f81b-bd68-4cce-afad-b9031d126015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:28.538392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:28.540296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:28.542272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:28.542272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:28.542272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:28.542272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:28.542272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 355/712] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:28.546309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f91ba-f2c2-4aeb-820d-0592cbe52999 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:28.699144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f91ba-f2c2-4aeb-820d-0592cbe52999 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:28.708003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:28.746088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7651de7-e05d-4d82-ac98-902562813980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:28.747314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7651de7-e05d-4d82-ac98-902562813980 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:28.747828 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7651de7-e05d-4d82-ac98-902562813980 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:28.748263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7651de7-e05d-4d82-ac98-902562813980 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:28.749057 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7651de7-e05d-4d82-ac98-902562813980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:28.749400 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7651de7-e05d-4d82-ac98-902562813980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:28.752947 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7651de7-e05d-4d82-ac98-902562813980 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:28.766927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 358/713] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:28.772747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8f81b-bd68-4cce-afad-b9031d126015 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:28.878685 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521842db-4941-472a-b06e-cf7c68cbf33f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:28.878959 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521842db-4941-472a-b06e-cf7c68cbf33f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:28.879293 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521842db-4941-472a-b06e-cf7c68cbf33f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:28.881119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 359/714] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:28.914667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd673f-3fd9-4647-abb0-ac1f93c5ded8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:28.914936 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd673f-3fd9-4647-abb0-ac1f93c5ded8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:28.915290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd673f-3fd9-4647-abb0-ac1f93c5ded8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:28.935432 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8f81b-bd68-4cce-afad-b9031d126015 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:28.940905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:28.957250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b5dd673f-3fd9-4647-abb0-ac1f93c5ded8 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:28.957250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b5dd673f-3fd9-4647-abb0-ac1f93c5ded8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:29.011113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2b6b7-332e-476f-8ddf-63cb13e91ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:29.012437 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2b6b7-332e-476f-8ddf-63cb13e91ce6 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:29.012741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2b6b7-332e-476f-8ddf-63cb13e91ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:29.013254 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2b6b7-332e-476f-8ddf-63cb13e91ce6 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:29.014505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2b6b7-332e-476f-8ddf-63cb13e91ce6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:29.017126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2b6b7-332e-476f-8ddf-63cb13e91ce6 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:29.082644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 356/715] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:22:28 2022] POST /identity/v3/users => generated 254 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:22:29.154007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 360/716] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:29.234339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2b6b7-332e-476f-8ddf-63cb13e91ce6 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:29.398549 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2b6b7-332e-476f-8ddf-63cb13e91ce6 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:29.446473 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c708a-6271-475f-bfd7-bf30b82d380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:29.447317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c708a-6271-475f-bfd7-bf30b82d380c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:29.447597 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c708a-6271-475f-bfd7-bf30b82d380c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:29.447874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c708a-6271-475f-bfd7-bf30b82d380c None admin] PATH_INFO: `/v3/services/network` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:29.448560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c708a-6271-475f-bfd7-bf30b82d380c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:29.450240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c708a-6271-475f-bfd7-bf30b82d380c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-878c708a-6271-475f-bfd7-bf30b82d380c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:29.466082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:29.468248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:29.470075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 357/717] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:29 2022] GET /identity/v3/services/network => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:29.475546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd673f-3fd9-4647-abb0-ac1f93c5ded8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:29.618656 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:29.660497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd673f-3fd9-4647-abb0-ac1f93c5ded8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:29.736530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb62947-0031-43da-9549-3d575e44ce05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:29.737728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb62947-0031-43da-9549-3d575e44ce05 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:29.738235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb62947-0031-43da-9549-3d575e44ce05 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:29.738739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb62947-0031-43da-9549-3d575e44ce05 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:29.740104 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb62947-0031-43da-9549-3d575e44ce05 None admin] RBAC: Adding query filter params (name=network) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:29.740668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb62947-0031-43da-9549-3d575e44ce05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:29.743377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb62947-0031-43da-9549-3d575e44ce05 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:29.752598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 361/718] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jun 7 08:22:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:29.763663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c708a-6271-475f-bfd7-bf30b82d380c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:29.825523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:29.959080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c708a-6271-475f-bfd7-bf30b82d380c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:30.013406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc184601-f6f7-420a-955f-4cd014f82e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:30.014094 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc184601-f6f7-420a-955f-4cd014f82e34 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:30.014376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc184601-f6f7-420a-955f-4cd014f82e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:30.014653 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc184601-f6f7-420a-955f-4cd014f82e34 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:30.015373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc184601-f6f7-420a-955f-4cd014f82e34 None admin] RBAC: Adding query filter params (type=network) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:30.015679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc184601-f6f7-420a-955f-4cd014f82e34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:30.017097 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc184601-f6f7-420a-955f-4cd014f82e34 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:30.028581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 358/719] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jun 7 08:22:29 2022] GET /identity/v3/services?type=network => generated 124 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:30.786495 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fee98c-99a4-47b2-b4cd-cbc602a955d9 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:30.787136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fee98c-99a4-47b2-b4cd-cbc602a955d9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:30.787136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fee98c-99a4-47b2-b4cd-cbc602a955d9 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:30.788911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 362/720] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:30.795272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8906bf-f155-4f0f-b141-dac2edaf2349 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:30.796259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8906bf-f155-4f0f-b141-dac2edaf2349 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:30.796767 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8906bf-f155-4f0f-b141-dac2edaf2349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:30.816687 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-db8906bf-f155-4f0f-b141-dac2edaf2349 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:30.818268 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-db8906bf-f155-4f0f-b141-dac2edaf2349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:30.928002 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 359/721] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:30.965028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fee98c-99a4-47b2-b4cd-cbc602a955d9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:31.011500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f09e3b-8af5-46e5-9cca-5e536d85c411 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:31.012367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f09e3b-8af5-46e5-9cca-5e536d85c411 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:31.012864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f09e3b-8af5-46e5-9cca-5e536d85c411 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:31.016155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 360/722] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:31.020969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f292a-bd45-4249-882d-b775cd1f09c4 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:31.021643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f292a-bd45-4249-882d-b775cd1f09c4 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:31.022141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f292a-bd45-4249-882d-b775cd1f09c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:31.057173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-4b0f292a-bd45-4249-882d-b775cd1f09c4 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:31.059074 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4b0f292a-bd45-4249-882d-b775cd1f09c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:31.084187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:31.144845 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fee98c-99a4-47b2-b4cd-cbc602a955d9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:31.197632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2b206-daf0-4fe1-85f4-bafdcc7544bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:31.200016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2b206-daf0-4fe1-85f4-bafdcc7544bb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:31.201035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2b206-daf0-4fe1-85f4-bafdcc7544bb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:31.201863 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2b206-daf0-4fe1-85f4-bafdcc7544bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:31.211560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 361/723] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:22:31.224813 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2b206-daf0-4fe1-85f4-bafdcc7544bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-dcd2b206-daf0-4fe1-85f4-bafdcc7544bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:31.241185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:31.243199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:31.244962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:31.244962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:31.244962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:31.244962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:31.244962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:31.244962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:31.244962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:31.244962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:31.244962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:31.244962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:31.244962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:31.244962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:31.247291 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f292a-bd45-4249-882d-b775cd1f09c4 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:31.250655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 363/724] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:31.270976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2b206-daf0-4fe1-85f4-bafdcc7544bb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:31.293247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:31.472135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2b206-daf0-4fe1-85f4-bafdcc7544bb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:31.554766 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f292a-bd45-4249-882d-b775cd1f09c4 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:31.562967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe4530-1364-4450-80fc-354f1d4a6127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:31.564101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe4530-1364-4450-80fc-354f1d4a6127 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:31.564612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe4530-1364-4450-80fc-354f1d4a6127 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:31.565130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe4530-1364-4450-80fc-354f1d4a6127 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:31.566542 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe4530-1364-4450-80fc-354f1d4a6127 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:31.567089 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe4530-1364-4450-80fc-354f1d4a6127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:31.569749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe4530-1364-4450-80fc-354f1d4a6127 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:31.604439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 364/725] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:31.610957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe4530-1364-4450-80fc-354f1d4a6127 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:31.681956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1dc0f-4060-4478-998b-1b2c0716e2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:31.683676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1dc0f-4060-4478-998b-1b2c0716e2ef None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:31.684585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1dc0f-4060-4478-998b-1b2c0716e2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:31.685222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1dc0f-4060-4478-998b-1b2c0716e2ef None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:31.686214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1dc0f-4060-4478-998b-1b2c0716e2ef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:31.687842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1dc0f-4060-4478-998b-1b2c0716e2ef None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:31.707591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 362/726] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:22:31 2022] POST /identity/v3/services => generated 251 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:22:31.709653 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8c87a-a737-46a8-ae36-d1705076c093 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:31.710231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8c87a-a737-46a8-ae36-d1705076c093 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:31.710733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8c87a-a737-46a8-ae36-d1705076c093 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:31.718997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 363/727] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:31.719697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51162fd6-e8fc-4cdf-8414-0c2306e77080 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:31.719995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51162fd6-e8fc-4cdf-8414-0c2306e77080 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:31.720279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51162fd6-e8fc-4cdf-8414-0c2306e77080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:31.739233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-51162fd6-e8fc-4cdf-8414-0c2306e77080 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:31.740429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-51162fd6-e8fc-4cdf-8414-0c2306e77080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:31.745412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:31.765922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:31.810776 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe4530-1364-4450-80fc-354f1d4a6127 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:31.847913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 364/728] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:22:31.880092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51162fd6-e8fc-4cdf-8414-0c2306e77080 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:31.887017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb074c5-c7f5-4a09-91bd-1d4bf8802118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:31.887622 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb074c5-c7f5-4a09-91bd-1d4bf8802118 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:31.887915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb074c5-c7f5-4a09-91bd-1d4bf8802118 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:31.888189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb074c5-c7f5-4a09-91bd-1d4bf8802118 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:31.920759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb074c5-c7f5-4a09-91bd-1d4bf8802118 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:31.922374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb074c5-c7f5-4a09-91bd-1d4bf8802118 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-3cb074c5-c7f5-4a09-91bd-1d4bf8802118 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:31.944647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:31.945676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:31.946583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:31.946583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:31.946583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:31.946583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:31.946583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 365/729] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:31.957824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb074c5-c7f5-4a09-91bd-1d4bf8802118 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:32.112911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb074c5-c7f5-4a09-91bd-1d4bf8802118 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:32.115690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51162fd6-e8fc-4cdf-8414-0c2306e77080 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:32.145915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:32.171978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c58535-d784-4b56-b064-b541c49df226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:32.172666 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c58535-d784-4b56-b064-b541c49df226 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:32.172987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c58535-d784-4b56-b064-b541c49df226 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:32.173255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c58535-d784-4b56-b064-b541c49df226 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:32.173993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c58535-d784-4b56-b064-b541c49df226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:32.174302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c58535-d784-4b56-b064-b541c49df226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:32.175705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c58535-d784-4b56-b064-b541c49df226 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:32.181465 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fb846-b314-438e-ad6a-1172188814fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:32.182307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fb846-b314-438e-ad6a-1172188814fe None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:32.182594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fb846-b314-438e-ad6a-1172188814fe None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:32.183033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fb846-b314-438e-ad6a-1172188814fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:32.185546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:32.203955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 366/730] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:32.205840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fb846-b314-438e-ad6a-1172188814fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:32.206396 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c58535-d784-4b56-b064-b541c49df226 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-df5fb846-b314-438e-ad6a-1172188814fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:32.215358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:32.216396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:32.217747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:32.217747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:32.217747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:32.217747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:32.217747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:32.217747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:32.217747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:32.217747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:32.217747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:32.217747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:32.217747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:32.217747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:32.221442 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 365/731] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:32.232125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fb846-b314-438e-ad6a-1172188814fe None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:32.363504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fb846-b314-438e-ad6a-1172188814fe None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:32.409443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d21aee-65d8-40e4-89a2-5f7d7df86160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:32.409443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d21aee-65d8-40e4-89a2-5f7d7df86160 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:32.410558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d21aee-65d8-40e4-89a2-5f7d7df86160 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:32.410558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d21aee-65d8-40e4-89a2-5f7d7df86160 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:32.410703 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d21aee-65d8-40e4-89a2-5f7d7df86160 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:32.411155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d21aee-65d8-40e4-89a2-5f7d7df86160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:32.412656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d21aee-65d8-40e4-89a2-5f7d7df86160 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:32.428424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 366/732] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:32.429183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c58535-d784-4b56-b064-b541c49df226 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:32.430141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8dd79-457d-4264-8d44-27864193f07d None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:32.430520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8dd79-457d-4264-8d44-27864193f07d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:32.430864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8dd79-457d-4264-8d44-27864193f07d None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:32.433412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 367/733] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:32.434134 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8dd79-457d-4264-8d44-27864193f07d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:32.477769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ce9ae-6a39-4b40-baa6-6b6dc15f313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:32.479311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ce9ae-6a39-4b40-baa6-6b6dc15f313b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:32.479796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ce9ae-6a39-4b40-baa6-6b6dc15f313b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:32.480861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ce9ae-6a39-4b40-baa6-6b6dc15f313b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:32.492239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ce9ae-6a39-4b40-baa6-6b6dc15f313b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:32.494371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ce9ae-6a39-4b40-baa6-6b6dc15f313b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:32.514699 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-9c0ce9ae-6a39-4b40-baa6-6b6dc15f313b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:32.517724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:32.518972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:32.519981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:32.519981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:32.519981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:32.519981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:32.519981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:32.519981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:32.519981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:32.519981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:32.519981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:22:32.519981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:32.523664 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 367/734] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:32.525287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb2ec4-ba27-4456-afba-708256738733 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:32.525769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb2ec4-ba27-4456-afba-708256738733 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:32.526213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb2ec4-ba27-4456-afba-708256738733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:32.551998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5adb2ec4-ba27-4456-afba-708256738733 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:32.553611 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5adb2ec4-ba27-4456-afba-708256738733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:32.600289 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8dd79-457d-4264-8d44-27864193f07d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:32.613642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:32.648384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd2bb5-654e-4d00-ab44-8906f9b0de8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:32.649452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd2bb5-654e-4d00-ab44-8906f9b0de8f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:32.649887 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd2bb5-654e-4d00-ab44-8906f9b0de8f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:32.650235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd2bb5-654e-4d00-ab44-8906f9b0de8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:32.652493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 368/735] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:32 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:32.655058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb2ec4-ba27-4456-afba-708256738733 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:32.665194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd2bb5-654e-4d00-ab44-8906f9b0de8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:32.668357 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd2bb5-654e-4d00-ab44-8906f9b0de8f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-62fd2bb5-654e-4d00-ab44-8906f9b0de8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:32.695220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:32.696201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:32.697547 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:32.697547 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:32.697547 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:32.697547 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:32.701480 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 368/736] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:32.702945 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd2bb5-654e-4d00-ab44-8906f9b0de8f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:32.795515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb2ec4-ba27-4456-afba-708256738733 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:32.844636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0e7e3-57a4-45cd-a503-83c48f147052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:32.846243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0e7e3-57a4-45cd-a503-83c48f147052 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:32.846736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0e7e3-57a4-45cd-a503-83c48f147052 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:32.847298 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0e7e3-57a4-45cd-a503-83c48f147052 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:32.848366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0e7e3-57a4-45cd-a503-83c48f147052 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:32.848894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0e7e3-57a4-45cd-a503-83c48f147052 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:32.850320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd2bb5-654e-4d00-ab44-8906f9b0de8f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:32.851146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0e7e3-57a4-45cd-a503-83c48f147052 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:32.861984 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:32.879845 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 369/737] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Jun 7 08:22:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:32.881357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0e7e3-57a4-45cd-a503-83c48f147052 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:32.894941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e41de-dc9d-47c3-85ab-17cf7141f994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:32.894941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e41de-dc9d-47c3-85ab-17cf7141f994 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:32.894941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e41de-dc9d-47c3-85ab-17cf7141f994 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:32.895320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e41de-dc9d-47c3-85ab-17cf7141f994 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:32.908002 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e41de-dc9d-47c3-85ab-17cf7141f994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-279e41de-dc9d-47c3-85ab-17cf7141f994 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:32.916812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:32.918435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:32.919275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:32.919275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:32.919275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:32.919275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:32.919275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:32.919275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:32.919275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:32.919275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:32.919275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:32.919275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:32.919275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:32.919275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:32.919937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 369/738] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:32.921017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e41de-dc9d-47c3-85ab-17cf7141f994 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:32.931682 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:33.043913 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0e7e3-57a4-45cd-a503-83c48f147052 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:33.063510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e41de-dc9d-47c3-85ab-17cf7141f994 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:33.082103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456899be-e88d-4fad-944d-2f72e3c4cee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:33.082775 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456899be-e88d-4fad-944d-2f72e3c4cee1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:33.083057 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456899be-e88d-4fad-944d-2f72e3c4cee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:33.083330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456899be-e88d-4fad-944d-2f72e3c4cee1 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:33.084066 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456899be-e88d-4fad-944d-2f72e3c4cee1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:33.084378 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456899be-e88d-4fad-944d-2f72e3c4cee1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:33.085785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456899be-e88d-4fad-944d-2f72e3c4cee1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:33.101324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 370/739] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:33.101324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456899be-e88d-4fad-944d-2f72e3c4cee1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:33.118390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d9e3a-0e1b-4984-a286-e4a548891568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:33.119357 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d9e3a-0e1b-4984-a286-e4a548891568 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:33.119823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d9e3a-0e1b-4984-a286-e4a548891568 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:33.120262 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d9e3a-0e1b-4984-a286-e4a548891568 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:33.184960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:33.228628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d9e3a-0e1b-4984-a286-e4a548891568 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:33.230749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d9e3a-0e1b-4984-a286-e4a548891568 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:33.232554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:33.246747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456899be-e88d-4fad-944d-2f72e3c4cee1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:33.296972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a27db0-dd02-41a1-98c5-67dd45ca8c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:33.298446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a27db0-dd02-41a1-98c5-67dd45ca8c61 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:33.298944 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a27db0-dd02-41a1-98c5-67dd45ca8c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:33.299865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a27db0-dd02-41a1-98c5-67dd45ca8c61 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:33.301766 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a27db0-dd02-41a1-98c5-67dd45ca8c61 None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:33.302689 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a27db0-dd02-41a1-98c5-67dd45ca8c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:33.304748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a27db0-dd02-41a1-98c5-67dd45ca8c61 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:33.319600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 371/740] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:33.321040 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc69ef-5ace-4952-bf0d-dc7657965014 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:33.321363 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc69ef-5ace-4952-bf0d-dc7657965014 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:33.321643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc69ef-5ace-4952-bf0d-dc7657965014 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:33.323270 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 372/741] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:33.324189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc69ef-5ace-4952-bf0d-dc7657965014 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:33.350363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 370/742] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:32 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:22:33.351760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d9e3a-0e1b-4984-a286-e4a548891568 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:33.440165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:33.478972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc69ef-5ace-4952-bf0d-dc7657965014 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:33.493164 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d9e3a-0e1b-4984-a286-e4a548891568 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:33.523188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1460d-73c4-4e9a-8246-03dec8396de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:33.523991 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1460d-73c4-4e9a-8246-03dec8396de7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:33.524278 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1460d-73c4-4e9a-8246-03dec8396de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:33.524651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1460d-73c4-4e9a-8246-03dec8396de7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:33.533433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1460d-73c4-4e9a-8246-03dec8396de7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:33.534887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1460d-73c4-4e9a-8246-03dec8396de7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:33.541250 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad710e-bf5d-493e-8a93-dd574f305359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:33.543042 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad710e-bf5d-493e-8a93-dd574f305359 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:33.543481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad710e-bf5d-493e-8a93-dd574f305359 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:33.543924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad710e-bf5d-493e-8a93-dd574f305359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-adb1460d-73c4-4e9a-8246-03dec8396de7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:33.550467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:33.551510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:33.552692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:33.552692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:33.552692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:33.552692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:33.552692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:33.552692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:33.552692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:33.552692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:33.552692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:22:33.552692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:33.556760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 373/743] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jun 7 08:22:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:33.558638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615cb9c-ca96-4792-866b-b6e1fd74661f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:33.558934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615cb9c-ca96-4792-866b-b6e1fd74661f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:33.559212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615cb9c-ca96-4792-866b-b6e1fd74661f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:33.560434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad710e-bf5d-493e-8a93-dd574f305359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:33.562369 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad710e-bf5d-493e-8a93-dd574f305359 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:33.578105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9615cb9c-ca96-4792-866b-b6e1fd74661f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:33.579583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9615cb9c-ca96-4792-866b-b6e1fd74661f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-bbad710e-bf5d-493e-8a93-dd574f305359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:33.584139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:33.585048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:33.586034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:33.586034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:33.586034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:33.586034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:33.592085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 371/744] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:33.592257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:33.592339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad710e-bf5d-493e-8a93-dd574f305359 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:33.672294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 374/745] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:33 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:22:33.672976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615cb9c-ca96-4792-866b-b6e1fd74661f None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:33.744477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad710e-bf5d-493e-8a93-dd574f305359 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:33.769635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:33.785617 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615cb9c-ca96-4792-866b-b6e1fd74661f None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:33.787424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485db96-c1ec-4fa1-8148-ba743160471e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:33.788551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485db96-c1ec-4fa1-8148-ba743160471e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:33.788677 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485db96-c1ec-4fa1-8148-ba743160471e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:33.788938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485db96-c1ec-4fa1-8148-ba743160471e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:33.801883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485db96-c1ec-4fa1-8148-ba743160471e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:33.803405 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485db96-c1ec-4fa1-8148-ba743160471e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-9485db96-c1ec-4fa1-8148-ba743160471e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:33.823517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:33.830189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:33.831088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:33.831088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:33.831088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:33.831088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:33.831307 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43352b65-2eb7-43b5-a2db-848456fc12cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:33.831406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 372/746] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:33.831476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43352b65-2eb7-43b5-a2db-848456fc12cb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:33.831476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43352b65-2eb7-43b5-a2db-848456fc12cb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:33.831476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43352b65-2eb7-43b5-a2db-848456fc12cb None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:33.831656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485db96-c1ec-4fa1-8148-ba743160471e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:33.831726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43352b65-2eb7-43b5-a2db-848456fc12cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:33.831726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43352b65-2eb7-43b5-a2db-848456fc12cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:33.831726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43352b65-2eb7-43b5-a2db-848456fc12cb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:33.847442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:33.852988 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 375/747] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jun 7 08:22:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:33.854031 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43352b65-2eb7-43b5-a2db-848456fc12cb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:33.981924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485db96-c1ec-4fa1-8148-ba743160471e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:34.003071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43352b65-2eb7-43b5-a2db-848456fc12cb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:34.038522 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c8a29-0caf-452e-82c1-8184dd8c6200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:34.039206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c8a29-0caf-452e-82c1-8184dd8c6200 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:34.039482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c8a29-0caf-452e-82c1-8184dd8c6200 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:34.039927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c8a29-0caf-452e-82c1-8184dd8c6200 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:34.040662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c8a29-0caf-452e-82c1-8184dd8c6200 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:34.041370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c8a29-0caf-452e-82c1-8184dd8c6200 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:34.043628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c8a29-0caf-452e-82c1-8184dd8c6200 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:34.049048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb94bb-fd22-4b37-96b9-67d12bfd971a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:34.049757 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb94bb-fd22-4b37-96b9-67d12bfd971a None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:34.050038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb94bb-fd22-4b37-96b9-67d12bfd971a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:34.050315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb94bb-fd22-4b37-96b9-67d12bfd971a None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:34.051051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb94bb-fd22-4b37-96b9-67d12bfd971a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:34.052518 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb94bb-fd22-4b37-96b9-67d12bfd971a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:34.063099 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 373/748] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:34.063099 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c8a29-0caf-452e-82c1-8184dd8c6200 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:34.072872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 376/749] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:22:33 2022] POST /identity/v3/services => generated 240 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:22:34.074526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb94bb-fd22-4b37-96b9-67d12bfd971a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:34.143858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:34.178733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:34.248484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb94bb-fd22-4b37-96b9-67d12bfd971a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:34.285140 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c8a29-0caf-452e-82c1-8184dd8c6200 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:34.293166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a6fcd-a990-4121-8bb1-9d22a1ef82e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:34.294338 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6fcd-a990-4121-8bb1-9d22a1ef82e6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:34.294720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6fcd-a990-4121-8bb1-9d22a1ef82e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:34.295098 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6fcd-a990-4121-8bb1-9d22a1ef82e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:34.309230 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a6fcd-a990-4121-8bb1-9d22a1ef82e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:34.311015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a6fcd-a990-4121-8bb1-9d22a1ef82e6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-248a6fcd-a990-4121-8bb1-9d22a1ef82e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:34.334101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:34.335068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:34.335987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:34.335987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:34.335987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:34.335987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:34.337014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 377/750] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:34.338050 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a6fcd-a990-4121-8bb1-9d22a1ef82e6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:34.338522 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce83a0-15e8-4754-90cb-437e1e2a43df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:34.339519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce83a0-15e8-4754-90cb-437e1e2a43df None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:34.340029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce83a0-15e8-4754-90cb-437e1e2a43df None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:34.340714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce83a0-15e8-4754-90cb-437e1e2a43df None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:34.341791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce83a0-15e8-4754-90cb-437e1e2a43df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:34.342658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce83a0-15e8-4754-90cb-437e1e2a43df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:34.344328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce83a0-15e8-4754-90cb-437e1e2a43df None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:34.358850 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 374/751] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:34.359645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce83a0-15e8-4754-90cb-437e1e2a43df None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:34.480326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a6fcd-a990-4121-8bb1-9d22a1ef82e6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:34.483760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:34.492641 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:34.499912 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce83a0-15e8-4754-90cb-437e1e2a43df None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:34.531741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18ce17-505e-4fc5-b125-c16f31f42ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:34.532650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18ce17-505e-4fc5-b125-c16f31f42ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:34.532944 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18ce17-505e-4fc5-b125-c16f31f42ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:34.533244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18ce17-505e-4fc5-b125-c16f31f42ae4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:34.539900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5800a-06b1-446f-9702-c27cfe44ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:34.540496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5800a-06b1-446f-9702-c27cfe44ae6a None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:34.540778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5800a-06b1-446f-9702-c27cfe44ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:34.541217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5800a-06b1-446f-9702-c27cfe44ae6a None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:34.542026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5800a-06b1-446f-9702-c27cfe44ae6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:34.542842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5800a-06b1-446f-9702-c27cfe44ae6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:34.543848 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18ce17-505e-4fc5-b125-c16f31f42ae4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:34.544474 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5800a-06b1-446f-9702-c27cfe44ae6a None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:34.545345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18ce17-505e-4fc5-b125-c16f31f42ae4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:34.561245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 375/752] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:34.561245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5800a-06b1-446f-9702-c27cfe44ae6a None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-1f18ce17-505e-4fc5-b125-c16f31f42ae4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:34.568441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:34.569486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:34.570440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:34.570440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:34.570440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:34.570440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:34.570440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:34.570440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:34.570440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:34.570440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:34.570440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:22:34.570440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:34.573435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 378/753] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jun 7 08:22:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:34.575663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e3ab2-2ea4-4775-91ef-d909f377e257 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:34.575956 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e3ab2-2ea4-4775-91ef-d909f377e257 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:34.576238 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e3ab2-2ea4-4775-91ef-d909f377e257 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:34.581709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 379/754] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:34.581709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e3ab2-2ea4-4775-91ef-d909f377e257 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:34.722331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e3ab2-2ea4-4775-91ef-d909f377e257 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:34.728189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5800a-06b1-446f-9702-c27cfe44ae6a None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:34.771078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920c5fd-7ff5-4909-bf2e-cae0c8b23bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:34.772259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920c5fd-7ff5-4909-bf2e-cae0c8b23bcf None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:34.772542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920c5fd-7ff5-4909-bf2e-cae0c8b23bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:34.772825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920c5fd-7ff5-4909-bf2e-cae0c8b23bcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:34.781654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920c5fd-7ff5-4909-bf2e-cae0c8b23bcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:34.783164 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920c5fd-7ff5-4909-bf2e-cae0c8b23bcf None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:34.794599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56967f7-76b1-4924-8e64-d5af186a1a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:34.795309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56967f7-76b1-4924-8e64-d5af186a1a6f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:34.795593 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56967f7-76b1-4924-8e64-d5af186a1a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:34.795867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56967f7-76b1-4924-8e64-d5af186a1a6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-1920c5fd-7ff5-4909-bf2e-cae0c8b23bcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:34.804044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:34.814264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:34.816350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:34.816350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:34.816350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:34.816350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:34.816350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:34.816350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:34.816350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:34.816350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:34.817799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56967f7-76b1-4924-8e64-d5af186a1a6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:34.817799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56967f7-76b1-4924-8e64-d5af186a1a6f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:34.818080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 376/755] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:34.818080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920c5fd-7ff5-4909-bf2e-cae0c8b23bcf None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:34.842058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-a56967f7-76b1-4924-8e64-d5af186a1a6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:34.843767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:34.844835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:34.848128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:34.848128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:34.848128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:34.848128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:34.848128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:34.848128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:34.848128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:34.848128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:34.852242 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 380/756] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:34.854331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b482267-b4cf-4fd1-94e1-cd001f7a59b6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:34.855279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b482267-b4cf-4fd1-94e1-cd001f7a59b6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:34.855877 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b482267-b4cf-4fd1-94e1-cd001f7a59b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:34.883421 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4b482267-b4cf-4fd1-94e1-cd001f7a59b6 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:34.885453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4b482267-b4cf-4fd1-94e1-cd001f7a59b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:34.926331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:35.003357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 381/757] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:35.004632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b482267-b4cf-4fd1-94e1-cd001f7a59b6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:35.015792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920c5fd-7ff5-4909-bf2e-cae0c8b23bcf None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:35.074568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8532317-ac77-446e-8bab-800e5fd98ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:35.075496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8532317-ac77-446e-8bab-800e5fd98ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:35.075946 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8532317-ac77-446e-8bab-800e5fd98ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:35.076389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8532317-ac77-446e-8bab-800e5fd98ab1 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:35.077879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8532317-ac77-446e-8bab-800e5fd98ab1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:35.078629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8532317-ac77-446e-8bab-800e5fd98ab1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:35.081114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8532317-ac77-446e-8bab-800e5fd98ab1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:35.119665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 377/758] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Jun 7 08:22:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:35.119665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8532317-ac77-446e-8bab-800e5fd98ab1 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:35.224240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:35.266439 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8532317-ac77-446e-8bab-800e5fd98ab1 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:35.321429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b482267-b4cf-4fd1-94e1-cd001f7a59b6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:35.323028 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1c6bc-298b-4072-a8da-4fd5ae5bf90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:35.324802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1c6bc-298b-4072-a8da-4fd5ae5bf90c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:35.325201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1c6bc-298b-4072-a8da-4fd5ae5bf90c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:35.325659 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1c6bc-298b-4072-a8da-4fd5ae5bf90c None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:35.327032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1c6bc-298b-4072-a8da-4fd5ae5bf90c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:35.327470 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1c6bc-298b-4072-a8da-4fd5ae5bf90c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:35.329990 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1c6bc-298b-4072-a8da-4fd5ae5bf90c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:35.353138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 378/759] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:35.353138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1c6bc-298b-4072-a8da-4fd5ae5bf90c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:35.387328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19e4f3-4095-45ad-a8f8-598934895ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:35.388068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19e4f3-4095-45ad-a8f8-598934895ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:35.388349 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19e4f3-4095-45ad-a8f8-598934895ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:35.388623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19e4f3-4095-45ad-a8f8-598934895ea7 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:35.390454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19e4f3-4095-45ad-a8f8-598934895ea7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:35.390766 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19e4f3-4095-45ad-a8f8-598934895ea7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:35.392462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19e4f3-4095-45ad-a8f8-598934895ea7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:35.408361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 382/760] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:35.411660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19e4f3-4095-45ad-a8f8-598934895ea7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:35.441673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:35.525301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1c6bc-298b-4072-a8da-4fd5ae5bf90c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:35.574697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:35.584833 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086cbd6b-e072-4f74-9ba0-68af9bd160c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:35.585887 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cbd6b-e072-4f74-9ba0-68af9bd160c8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:35.586343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cbd6b-e072-4f74-9ba0-68af9bd160c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:35.586800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cbd6b-e072-4f74-9ba0-68af9bd160c8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:35.587606 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086cbd6b-e072-4f74-9ba0-68af9bd160c8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:35.587859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19e4f3-4095-45ad-a8f8-598934895ea7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:35.589303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086cbd6b-e072-4f74-9ba0-68af9bd160c8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-086cbd6b-e072-4f74-9ba0-68af9bd160c8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:35.612997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:35.615101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:22:35.616733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:35.625119 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 379/761] 10.222.0.39 () {56 vars in 1129 bytes} [Tue Jun 7 08:22:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:35.626274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086cbd6b-e072-4f74-9ba0-68af9bd160c8 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:35.643232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e60b7-5234-4a80-9b7d-079172264403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:35.643874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e60b7-5234-4a80-9b7d-079172264403 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:35.644163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e60b7-5234-4a80-9b7d-079172264403 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:35.644547 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e60b7-5234-4a80-9b7d-079172264403 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:35.748526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e60b7-5234-4a80-9b7d-079172264403 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:35.750907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e60b7-5234-4a80-9b7d-079172264403 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:35.763205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086cbd6b-e072-4f74-9ba0-68af9bd160c8 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:35.775012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:35.811642 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994d8ef-6ee7-47f4-b708-e70bce15a09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:35.812800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994d8ef-6ee7-47f4-b708-e70bce15a09e None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:35.813144 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994d8ef-6ee7-47f4-b708-e70bce15a09e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:35.813406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994d8ef-6ee7-47f4-b708-e70bce15a09e None admin] PATH_INFO: `/v3/projects/07b861f95bf3498d990785ba72474d82/users/3f2a2bf913d1412f996acf4401ba7673/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:35.815620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994d8ef-6ee7-47f4-b708-e70bce15a09e None admin] RBAC: Authorizing `identity:create_grant(project_id=07b861f95bf3498d990785ba72474d82, user_id=3f2a2bf913d1412f996acf4401ba7673, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:35.817110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994d8ef-6ee7-47f4-b708-e70bce15a09e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:35.834214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 380/762] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:22:35 2022] PUT /identity/v3/projects/07b861f95bf3498d990785ba72474d82/users/3f2a2bf913d1412f996acf4401ba7673/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:22:35.835143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994d8ef-6ee7-47f4-b708-e70bce15a09e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:35.917294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 383/763] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:22:35.920226 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e60b7-5234-4a80-9b7d-079172264403 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:35.936966 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:36.004387 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994d8ef-6ee7-47f4-b708-e70bce15a09e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:36.066516 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bedd7-8057-4de3-9449-7360d024ad0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:36.068868 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bedd7-8057-4de3-9449-7360d024ad0e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:36.069637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bedd7-8057-4de3-9449-7360d024ad0e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:36.070422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bedd7-8057-4de3-9449-7360d024ad0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:36.072203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bedd7-8057-4de3-9449-7360d024ad0e None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=586ddffa77a54521b42b2aa50f16cb92) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:36.072999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bedd7-8057-4de3-9449-7360d024ad0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:36.076745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bedd7-8057-4de3-9449-7360d024ad0e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:36.088416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 381/764] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:35 2022] GET /identity/v3/role_assignments?user.id=586ddffa77a54521b42b2aa50f16cb92&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:36.090447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bedd7-8057-4de3-9449-7360d024ad0e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:36.103430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e60b7-5234-4a80-9b7d-079172264403 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:36.138289 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:36.195226 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf23a56-2c01-4590-aa7f-5aa752d823f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:36.196317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf23a56-2c01-4590-aa7f-5aa752d823f3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:36.196804 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf23a56-2c01-4590-aa7f-5aa752d823f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:36.197314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf23a56-2c01-4590-aa7f-5aa752d823f3 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:36.199442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf23a56-2c01-4590-aa7f-5aa752d823f3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:36.199872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf23a56-2c01-4590-aa7f-5aa752d823f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:36.202664 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf23a56-2c01-4590-aa7f-5aa752d823f3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:36.230342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 384/765] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Jun 7 08:22:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:36.239772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf23a56-2c01-4590-aa7f-5aa752d823f3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:36.307361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bedd7-8057-4de3-9449-7360d024ad0e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:36.349585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:36.377202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4efdb-8457-4f6a-9ca1-691496009fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:36.378362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4efdb-8457-4f6a-9ca1-691496009fba None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:36.378868 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4efdb-8457-4f6a-9ca1-691496009fba None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:36.379361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4efdb-8457-4f6a-9ca1-691496009fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:36.395315 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4efdb-8457-4f6a-9ca1-691496009fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:36.397439 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4efdb-8457-4f6a-9ca1-691496009fba None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-4df4efdb-8457-4f6a-9ca1-691496009fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:36.421654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:36.422619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:36.423593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:36.423593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:36.423593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:36.423593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:36.426087 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 382/766] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:36.451043 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4efdb-8457-4f6a-9ca1-691496009fba None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:36.474750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf23a56-2c01-4590-aa7f-5aa752d823f3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:36.549132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbd556-9ad0-45a4-90fd-afc5ad5b37de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:36.549132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbd556-9ad0-45a4-90fd-afc5ad5b37de None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:36.549132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbd556-9ad0-45a4-90fd-afc5ad5b37de None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:36.549132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbd556-9ad0-45a4-90fd-afc5ad5b37de None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:36.549784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbd556-9ad0-45a4-90fd-afc5ad5b37de None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:36.549784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbd556-9ad0-45a4-90fd-afc5ad5b37de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:36.551213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbd556-9ad0-45a4-90fd-afc5ad5b37de None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:36.562088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 385/767] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Jun 7 08:22:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:36.576552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbd556-9ad0-45a4-90fd-afc5ad5b37de None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:36.661996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:36.704706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4efdb-8457-4f6a-9ca1-691496009fba None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:36.743665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea98c0-70db-4107-9e3a-cb4095d966ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:36.744344 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea98c0-70db-4107-9e3a-cb4095d966ab None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:36.744634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea98c0-70db-4107-9e3a-cb4095d966ab None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:36.744925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea98c0-70db-4107-9e3a-cb4095d966ab None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:36.745744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea98c0-70db-4107-9e3a-cb4095d966ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:36.746055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea98c0-70db-4107-9e3a-cb4095d966ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:36.747532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea98c0-70db-4107-9e3a-cb4095d966ab None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:36.760522 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 383/768] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:36.765524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e64e-c536-42b6-9b46-22c3e4b9b6b4 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:36.765524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e64e-c536-42b6-9b46-22c3e4b9b6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:36.765524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e64e-c536-42b6-9b46-22c3e4b9b6b4 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:36.765524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 384/769] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:36.766090 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbd556-9ad0-45a4-90fd-afc5ad5b37de None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:36.769756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6586501-4ba1-4731-886e-73ff67d4b13f None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:36.770139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6586501-4ba1-4731-886e-73ff67d4b13f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:36.770297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6586501-4ba1-4731-886e-73ff67d4b13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:36.798558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e6586501-4ba1-4731-886e-73ff67d4b13f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:36.800953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e6586501-4ba1-4731-886e-73ff67d4b13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:36.809879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac16af-d96d-4fc9-8c81-bcc3a93cfb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:36.810654 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac16af-d96d-4fc9-8c81-bcc3a93cfb9e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:36.811008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac16af-d96d-4fc9-8c81-bcc3a93cfb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:36.811353 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac16af-d96d-4fc9-8c81-bcc3a93cfb9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:36.812225 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac16af-d96d-4fc9-8c81-bcc3a93cfb9e None admin] RBAC: Adding query filter params (interface=public, service_id=a2d0368a1b8d43e6b11938919b082ce0, region_id=RegionOne) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:36.812609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac16af-d96d-4fc9-8c81-bcc3a93cfb9e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:36.814240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac16af-d96d-4fc9-8c81-bcc3a93cfb9e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:36.831351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 386/770] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jun 7 08:22:36 2022] GET /identity/v3/endpoints?service_id=a2d0368a1b8d43e6b11938919b082ce0&interface=public®ion_id=RegionOne => generated 194 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:36.831351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac16af-d96d-4fc9-8c81-bcc3a93cfb9e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:36.847415 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:36.898080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 385/771] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:22:36.898707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6586501-4ba1-4731-886e-73ff67d4b13f None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:37.013727 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6586501-4ba1-4731-886e-73ff67d4b13f None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:37.020319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac16af-d96d-4fc9-8c81-bcc3a93cfb9e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:37.040992 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:37.057136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323c17d-fe7a-482d-9e47-e16d4d7e7917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:37.058410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323c17d-fe7a-482d-9e47-e16d4d7e7917 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:37.058702 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323c17d-fe7a-482d-9e47-e16d4d7e7917 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:37.059088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323c17d-fe7a-482d-9e47-e16d4d7e7917 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:37.060093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323c17d-fe7a-482d-9e47-e16d4d7e7917 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:37.061831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323c17d-fe7a-482d-9e47-e16d4d7e7917 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:37.073127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 386/772] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jun 7 08:22:36 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:37.073127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323c17d-fe7a-482d-9e47-e16d4d7e7917 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:37.114513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2cbaf-275c-4ee9-8051-388a2e1adbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:37.115329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2cbaf-275c-4ee9-8051-388a2e1adbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:37.115663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2cbaf-275c-4ee9-8051-388a2e1adbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:37.115965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2cbaf-275c-4ee9-8051-388a2e1adbb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:37.136898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2cbaf-275c-4ee9-8051-388a2e1adbb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:37.140154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2cbaf-275c-4ee9-8051-388a2e1adbb2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:37.145924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-69e2cbaf-275c-4ee9-8051-388a2e1adbb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:37.185216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:37.189162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:37.192574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:37.192574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:37.192574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:37.192574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:37.192574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:37.192574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:37.192574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:37.192574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:37.192574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 387/773] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:37.203891 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2cbaf-275c-4ee9-8051-388a2e1adbb2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:37.324022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323c17d-fe7a-482d-9e47-e16d4d7e7917 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:37.412177 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98075db-123d-41db-9bb9-584de155cb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:37.412177 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98075db-123d-41db-9bb9-584de155cb55 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:37.412177 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98075db-123d-41db-9bb9-584de155cb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:37.412177 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98075db-123d-41db-9bb9-584de155cb55 None admin] PATH_INFO: `/v3/services/network` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:37.412800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98075db-123d-41db-9bb9-584de155cb55 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:37.412800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98075db-123d-41db-9bb9-584de155cb55 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:37.427261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2cbaf-275c-4ee9-8051-388a2e1adbb2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-f98075db-123d-41db-9bb9-584de155cb55 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:37.443355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:37.445160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:22:37.446634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:37.453842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 387/774] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:37 2022] GET /identity/v3/services/network => generated 88 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:37.460343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98075db-123d-41db-9bb9-584de155cb55 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:37.498929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda619e-2f06-4d49-b22d-227451135728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:37.500705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda619e-2f06-4d49-b22d-227451135728 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:37.501542 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda619e-2f06-4d49-b22d-227451135728 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:37.502295 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda619e-2f06-4d49-b22d-227451135728 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:37.504084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda619e-2f06-4d49-b22d-227451135728 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:37.504889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda619e-2f06-4d49-b22d-227451135728 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:37.507682 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda619e-2f06-4d49-b22d-227451135728 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:37.528986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 388/775] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:37.538710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda619e-2f06-4d49-b22d-227451135728 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:37.621512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:37.650529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:37.665594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98075db-123d-41db-9bb9-584de155cb55 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:37.740708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f333c2-cbef-4e67-85f0-6d735ca4e4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:37.741428 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f333c2-cbef-4e67-85f0-6d735ca4e4ea None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:37.741719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f333c2-cbef-4e67-85f0-6d735ca4e4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:37.742001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f333c2-cbef-4e67-85f0-6d735ca4e4ea None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:37.742923 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f333c2-cbef-4e67-85f0-6d735ca4e4ea None admin] RBAC: Adding query filter params (name=network) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:37.743458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f333c2-cbef-4e67-85f0-6d735ca4e4ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:37.744922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f333c2-cbef-4e67-85f0-6d735ca4e4ea None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:37.755647 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 388/776] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jun 7 08:22:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:37.759620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda619e-2f06-4d49-b22d-227451135728 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:37.762572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f333c2-cbef-4e67-85f0-6d735ca4e4ea None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:37.852637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360d028-6af8-48e0-a329-014243f6d29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:37.853376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360d028-6af8-48e0-a329-014243f6d29e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:37.853748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360d028-6af8-48e0-a329-014243f6d29e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:37.854028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360d028-6af8-48e0-a329-014243f6d29e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:37.855594 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360d028-6af8-48e0-a329-014243f6d29e None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=26d6eb06b23b4db88261ae58d650736d) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:37.855923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360d028-6af8-48e0-a329-014243f6d29e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:37.858202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360d028-6af8-48e0-a329-014243f6d29e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:37.874532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 389/777] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:37 2022] GET /identity/v3/role_assignments?user.id=26d6eb06b23b4db88261ae58d650736d&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 259 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:37.926471 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f333c2-cbef-4e67-85f0-6d735ca4e4ea None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:37.967034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ab406-8dbf-4081-93a3-39a16b38db2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:37.967722 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ab406-8dbf-4081-93a3-39a16b38db2e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:37.968009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ab406-8dbf-4081-93a3-39a16b38db2e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:37.968285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ab406-8dbf-4081-93a3-39a16b38db2e None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:37.969004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ab406-8dbf-4081-93a3-39a16b38db2e None admin] RBAC: Adding query filter params (type=network) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:37.969340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ab406-8dbf-4081-93a3-39a16b38db2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:37.970785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ab406-8dbf-4081-93a3-39a16b38db2e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:37.977053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 389/778] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jun 7 08:22:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:37.983826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360d028-6af8-48e0-a329-014243f6d29e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:38.194183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:38.209562 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360d028-6af8-48e0-a329-014243f6d29e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:38.269995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621da073-f373-48e9-ba11-e97526e6e95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:38.270985 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621da073-f373-48e9-ba11-e97526e6e95b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:38.271407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621da073-f373-48e9-ba11-e97526e6e95b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:38.271820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621da073-f373-48e9-ba11-e97526e6e95b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:38.273067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621da073-f373-48e9-ba11-e97526e6e95b None admin] RBAC: Adding query filter params (interface=public, service_id=b89b997868404ff1b09ba63d961528b1, region_id=RegionOne) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:38.273599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621da073-f373-48e9-ba11-e97526e6e95b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:38.275719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621da073-f373-48e9-ba11-e97526e6e95b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:38.286324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 390/779] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jun 7 08:22:37 2022] GET /identity/v3/endpoints?service_id=b89b997868404ff1b09ba63d961528b1&interface=public®ion_id=RegionOne => generated 194 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:38.294247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ab406-8dbf-4081-93a3-39a16b38db2e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:38.329129 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:38.446682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ab406-8dbf-4081-93a3-39a16b38db2e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:38.483474 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3ffbf-b2f9-41b5-8153-c800a8d00f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:38.484150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3ffbf-b2f9-41b5-8153-c800a8d00f1d None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:38.484434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3ffbf-b2f9-41b5-8153-c800a8d00f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:38.484711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3ffbf-b2f9-41b5-8153-c800a8d00f1d None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:38.485420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3ffbf-b2f9-41b5-8153-c800a8d00f1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:38.486802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3ffbf-b2f9-41b5-8153-c800a8d00f1d None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:38.492744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 390/780] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jun 7 08:22:38 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:39.187318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcb2c8-c313-4f8e-8959-843ce269fdb9 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:39.188098 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcb2c8-c313-4f8e-8959-843ce269fdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:39.188664 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcb2c8-c313-4f8e-8959-843ce269fdb9 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:39.191151 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 391/781] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:39.196529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4d01e-ccaf-432e-b269-0778a3ea49bd None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:39.197105 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4d01e-ccaf-432e-b269-0778a3ea49bd None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:39.197584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4d01e-ccaf-432e-b269-0778a3ea49bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:39.226852 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-5fb4d01e-ccaf-432e-b269-0778a3ea49bd None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:39.228489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5fb4d01e-ccaf-432e-b269-0778a3ea49bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:39.374800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 391/782] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:39.387708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15bcc8-3844-4088-a979-944d1de3ef09 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:39.389352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15bcc8-3844-4088-a979-944d1de3ef09 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:39.390070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15bcc8-3844-4088-a979-944d1de3ef09 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:39.392709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 392/783] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:39.399727 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a715c50-4161-4079-9a99-bea6ef8e28cd None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:39.400500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a715c50-4161-4079-9a99-bea6ef8e28cd None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:39.400500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a715c50-4161-4079-9a99-bea6ef8e28cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:39.426359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-0a715c50-4161-4079-9a99-bea6ef8e28cd None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:39.427695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0a715c50-4161-4079-9a99-bea6ef8e28cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:39.436543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15bcc8-3844-4088-a979-944d1de3ef09 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:39.437404 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:39.565608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15bcc8-3844-4088-a979-944d1de3ef09 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:39.604061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a95218-8cbe-4be8-92c0-537970e11fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:39.605427 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a95218-8cbe-4be8-92c0-537970e11fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:39.605918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a95218-8cbe-4be8-92c0-537970e11fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:39.606393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a95218-8cbe-4be8-92c0-537970e11fd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:39.619784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 392/784] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:22:39.621127 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a95218-8cbe-4be8-92c0-537970e11fd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-e7a95218-8cbe-4be8-92c0-537970e11fd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:39.629169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:39.630126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:39.630945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:39.630945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:39.630945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:39.630945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:39.630945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:39.630945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:39.630945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:39.630945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:39.630945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:39.630945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:39.630945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:39.630945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:39.633810 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 393/785] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:39.643232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a715c50-4161-4079-9a99-bea6ef8e28cd None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:39.657210 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a95218-8cbe-4be8-92c0-537970e11fd2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:39.681730 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:39.856467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a95218-8cbe-4be8-92c0-537970e11fd2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:39.923285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130f664-e1f8-430f-a859-5938505b8d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:39.925224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130f664-e1f8-430f-a859-5938505b8d9e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:39.926000 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130f664-e1f8-430f-a859-5938505b8d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:39.926795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130f664-e1f8-430f-a859-5938505b8d9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:39.947438 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130f664-e1f8-430f-a859-5938505b8d9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-0130f664-e1f8-430f-a859-5938505b8d9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:39.966464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:39.968362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:39.970086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:39.970086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:39.970086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:39.970086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:39.970086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:39.970086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:39.970086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:39.970086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:39.970086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:39.970086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:39.970086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:39.970086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:39.971388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a715c50-4161-4079-9a99-bea6ef8e28cd None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:39.975337 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 394/786] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:39.986032 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130f664-e1f8-430f-a859-5938505b8d9e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:40.062693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686401d8-d3c7-424c-a353-ba6fdc2cad96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:40.063395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686401d8-d3c7-424c-a353-ba6fdc2cad96 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:40.063734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686401d8-d3c7-424c-a353-ba6fdc2cad96 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:40.064020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686401d8-d3c7-424c-a353-ba6fdc2cad96 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:40.064797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686401d8-d3c7-424c-a353-ba6fdc2cad96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:40.065131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686401d8-d3c7-424c-a353-ba6fdc2cad96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:40.066710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686401d8-d3c7-424c-a353-ba6fdc2cad96 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:40.097058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 393/787] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:40.106851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686401d8-d3c7-424c-a353-ba6fdc2cad96 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:40.111626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:40.216254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:40.279995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130f664-e1f8-430f-a859-5938505b8d9e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:40.287905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686401d8-d3c7-424c-a353-ba6fdc2cad96 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:40.336353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c0e65-0865-4228-a147-cafb77deff29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:40.337500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c0e65-0865-4228-a147-cafb77deff29 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:40.337862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c0e65-0865-4228-a147-cafb77deff29 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:40.338302 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c0e65-0865-4228-a147-cafb77deff29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:40.369231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2758d-dcd8-4497-ae02-d93348ba460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:40.369930 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2758d-dcd8-4497-ae02-d93348ba460d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:40.371679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c0e65-0865-4228-a147-cafb77deff29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:40.373332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c0e65-0865-4228-a147-cafb77deff29 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:40.373699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2758d-dcd8-4497-ae02-d93348ba460d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:40.374015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2758d-dcd8-4497-ae02-d93348ba460d None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:40.375302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2758d-dcd8-4497-ae02-d93348ba460d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:40.375821 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2758d-dcd8-4497-ae02-d93348ba460d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:40.377528 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2758d-dcd8-4497-ae02-d93348ba460d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-690c0e65-0865-4228-a147-cafb77deff29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:40.414306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:40.416048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:40.416980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:40.416980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:40.416980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:40.416980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:40.417531 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 395/788] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:40.420727 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 394/789] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:40.425910 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2758d-dcd8-4497-ae02-d93348ba460d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:40.427404 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c0e65-0865-4228-a147-cafb77deff29 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:40.588766 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:40.615732 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2758d-dcd8-4497-ae02-d93348ba460d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:40.671659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4f7c5-65ab-4990-8080-38b254f3f947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:40.672515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4f7c5-65ab-4990-8080-38b254f3f947 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:40.672863 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4f7c5-65ab-4990-8080-38b254f3f947 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:40.673228 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4f7c5-65ab-4990-8080-38b254f3f947 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:40.674683 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4f7c5-65ab-4990-8080-38b254f3f947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:40.675064 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4f7c5-65ab-4990-8080-38b254f3f947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:40.676973 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4f7c5-65ab-4990-8080-38b254f3f947 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:40.691684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:40.696557 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 396/790] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:40.699422 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceee0b-67df-492c-8570-d0719e605e54 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:40.699948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceee0b-67df-492c-8570-d0719e605e54 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:40.700415 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceee0b-67df-492c-8570-d0719e605e54 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:40.700918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c0e65-0865-4228-a147-cafb77deff29 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:40.702455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 397/791] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:40.703695 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ceee0b-67df-492c-8570-d0719e605e54 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:40.753226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ccc05-293d-4273-91d2-16057226b931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:40.754530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ccc05-293d-4273-91d2-16057226b931 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:40.755023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ccc05-293d-4273-91d2-16057226b931 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:40.755533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ccc05-293d-4273-91d2-16057226b931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:40.769492 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ccc05-293d-4273-91d2-16057226b931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:40.771434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ccc05-293d-4273-91d2-16057226b931 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-203ccc05-293d-4273-91d2-16057226b931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:40.791253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:40.792760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:40.793694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:40.793694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:40.793694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:40.793694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:40.796073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 395/792] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:40.797784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c06db-d742-4190-b7ea-049577d2432b None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:40.798294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c06db-d742-4190-b7ea-049577d2432b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:40.798794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c06db-d742-4190-b7ea-049577d2432b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:40.826758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-cf3c06db-d742-4190-b7ea-049577d2432b None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:40.829467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-cf3c06db-d742-4190-b7ea-049577d2432b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:40.847222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ceee0b-67df-492c-8570-d0719e605e54 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:40.894306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af8e6a-5409-4ced-ba7a-0ccfe8900706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:40.895178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af8e6a-5409-4ced-ba7a-0ccfe8900706 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:40.895524 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af8e6a-5409-4ced-ba7a-0ccfe8900706 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:40.895801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af8e6a-5409-4ced-ba7a-0ccfe8900706 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:40.905549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af8e6a-5409-4ced-ba7a-0ccfe8900706 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:40.907464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af8e6a-5409-4ced-ba7a-0ccfe8900706 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:40.930205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 396/793] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:40.931410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c06db-d742-4190-b7ea-049577d2432b None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f2af8e6a-5409-4ced-ba7a-0ccfe8900706 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:40.946079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:40.947114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:40.948084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:40.948084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:40.948084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:40.948084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:40.948084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:40.948084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:40.948084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:40.948084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:40.948084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:22:40.948084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:40.949669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 398/794] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:40.955496 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af8e6a-5409-4ced-ba7a-0ccfe8900706 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:40.975441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:41.062662 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:41.083138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c06db-d742-4190-b7ea-049577d2432b None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:41.130527 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af8e6a-5409-4ced-ba7a-0ccfe8900706 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:41.143850 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eff160-d334-4354-83c8-574527d3bd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:41.144517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eff160-d334-4354-83c8-574527d3bd04 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:41.144798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eff160-d334-4354-83c8-574527d3bd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:41.145433 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eff160-d334-4354-83c8-574527d3bd04 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:41.145821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eff160-d334-4354-83c8-574527d3bd04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:41.146125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eff160-d334-4354-83c8-574527d3bd04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:41.147747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eff160-d334-4354-83c8-574527d3bd04 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:41.167630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 397/795] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:41.167630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56610a-9ab9-4129-bc35-c3fc5c54e799 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:41.167630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56610a-9ab9-4129-bc35-c3fc5c54e799 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:41.167630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56610a-9ab9-4129-bc35-c3fc5c54e799 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:41.171089 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 398/796] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:41.172550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56610a-9ab9-4129-bc35-c3fc5c54e799 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:41.195435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ca61d-a4a4-4feb-978c-ef1f73d9d11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:41.196364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ca61d-a4a4-4feb-978c-ef1f73d9d11a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:41.196792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ca61d-a4a4-4feb-978c-ef1f73d9d11a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:41.197231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ca61d-a4a4-4feb-978c-ef1f73d9d11a None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:41.199181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ca61d-a4a4-4feb-978c-ef1f73d9d11a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:41.199669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ca61d-a4a4-4feb-978c-ef1f73d9d11a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:41.202004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ca61d-a4a4-4feb-978c-ef1f73d9d11a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:41.231506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 399/797] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Jun 7 08:22:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:41.232658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ca61d-a4a4-4feb-978c-ef1f73d9d11a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:41.300831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:41.314018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56610a-9ab9-4129-bc35-c3fc5c54e799 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:41.358204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83eefa9-7b23-46a1-8705-46e4f15fd45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:41.359026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83eefa9-7b23-46a1-8705-46e4f15fd45c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:41.359317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83eefa9-7b23-46a1-8705-46e4f15fd45c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:41.359597 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83eefa9-7b23-46a1-8705-46e4f15fd45c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:41.360265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83eefa9-7b23-46a1-8705-46e4f15fd45c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:41.361706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ca61d-a4a4-4feb-978c-ef1f73d9d11a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:41.361888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83eefa9-7b23-46a1-8705-46e4f15fd45c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-b83eefa9-7b23-46a1-8705-46e4f15fd45c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:41.376232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:41.377775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:41.379283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 399/798] 10.222.0.39 () {56 vars in 1129 bytes} [Tue Jun 7 08:22:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:41.380167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68983cd1-8795-4d48-bc7e-c2868829f892 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:41.380167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68983cd1-8795-4d48-bc7e-c2868829f892 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:41.380167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68983cd1-8795-4d48-bc7e-c2868829f892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:41.389317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:41.404156 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-68983cd1-8795-4d48-bc7e-c2868829f892 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:41.406311 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-68983cd1-8795-4d48-bc7e-c2868829f892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:41.408408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ec8ff-549b-49b3-a471-9a0f880a339f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:41.409270 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ec8ff-549b-49b3-a471-9a0f880a339f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:41.409552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ec8ff-549b-49b3-a471-9a0f880a339f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:41.409868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ec8ff-549b-49b3-a471-9a0f880a339f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:41.419634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ec8ff-549b-49b3-a471-9a0f880a339f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:41.420769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ec8ff-549b-49b3-a471-9a0f880a339f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-1c7ec8ff-549b-49b3-a471-9a0f880a339f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:41.437265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:41.438294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:41.444052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:41.444052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:41.444052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:41.444052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:41.444052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:41.444052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:41.444052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:41.444052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:41.444052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:22:41.444052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:41.444052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 400/799] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jun 7 08:22:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:41.444052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ec8ff-549b-49b3-a471-9a0f880a339f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:41.525267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 400/800] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:41.525267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68983cd1-8795-4d48-bc7e-c2868829f892 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:41.582472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ec8ff-549b-49b3-a471-9a0f880a339f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:41.620740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c2b14-609c-451c-8ba0-883f81803748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:41.621609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c2b14-609c-451c-8ba0-883f81803748 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:41.621894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c2b14-609c-451c-8ba0-883f81803748 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:41.622174 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c2b14-609c-451c-8ba0-883f81803748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:41.636589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c2b14-609c-451c-8ba0-883f81803748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:41.638860 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c2b14-609c-451c-8ba0-883f81803748 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-9a9c2b14-609c-451c-8ba0-883f81803748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:41.665359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:41.666479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:41.668560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:41.668560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:41.668560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:41.668560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:41.668560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 401/801] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:41.668560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c2b14-609c-451c-8ba0-883f81803748 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:41.685886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:41.702122 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68983cd1-8795-4d48-bc7e-c2868829f892 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:41.705812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:41.741259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b22d9-56f1-47b7-9e9c-cd57038b7260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:41.741941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b22d9-56f1-47b7-9e9c-cd57038b7260 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:41.742221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b22d9-56f1-47b7-9e9c-cd57038b7260 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:41.742489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b22d9-56f1-47b7-9e9c-cd57038b7260 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:41.743205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b22d9-56f1-47b7-9e9c-cd57038b7260 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:41.743507 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b22d9-56f1-47b7-9e9c-cd57038b7260 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:41.744948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b22d9-56f1-47b7-9e9c-cd57038b7260 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:41.751261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 401/802] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Jun 7 08:22:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:41.752587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b22d9-56f1-47b7-9e9c-cd57038b7260 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:41.790609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c2b14-609c-451c-8ba0-883f81803748 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:41.836214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791b75f-d7fd-4258-bbd0-ee083360dec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:41.836623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791b75f-d7fd-4258-bbd0-ee083360dec2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:41.836909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791b75f-d7fd-4258-bbd0-ee083360dec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:41.837230 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791b75f-d7fd-4258-bbd0-ee083360dec2 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:41.838092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791b75f-d7fd-4258-bbd0-ee083360dec2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:41.838408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791b75f-d7fd-4258-bbd0-ee083360dec2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:41.839939 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791b75f-d7fd-4258-bbd0-ee083360dec2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:41.859735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 402/803] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jun 7 08:22:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:41.860403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791b75f-d7fd-4258-bbd0-ee083360dec2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:41.873839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b22d9-56f1-47b7-9e9c-cd57038b7260 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:41.924017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd3069-ea21-44ff-9b41-53316a7f4d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:41.924822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd3069-ea21-44ff-9b41-53316a7f4d57 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:41.925271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd3069-ea21-44ff-9b41-53316a7f4d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:41.925467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd3069-ea21-44ff-9b41-53316a7f4d57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:41.937690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd3069-ea21-44ff-9b41-53316a7f4d57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-e2fd3069-ea21-44ff-9b41-53316a7f4d57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:41.945593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:41.946620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:41.947451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:41.947451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:41.947451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:41.947451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:41.947451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:41.947451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:41.947451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:41.947451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:41.947451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:41.947451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:41.947451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:41.947451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:41.947451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 402/804] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:41.948131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d8e38-53af-4b28-9602-f4143505c906 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:41.948502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d8e38-53af-4b28-9602-f4143505c906 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:41.948847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d8e38-53af-4b28-9602-f4143505c906 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:41.950292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 403/805] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:41.951270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d8e38-53af-4b28-9602-f4143505c906 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:41.971005 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:41.971225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:41.985467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791b75f-d7fd-4258-bbd0-ee083360dec2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:42.023155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060d476-cfad-4ac3-9860-f707a6673e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:42.023829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060d476-cfad-4ac3-9860-f707a6673e6b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:42.024120 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060d476-cfad-4ac3-9860-f707a6673e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:42.024400 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060d476-cfad-4ac3-9860-f707a6673e6b None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:42.025172 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060d476-cfad-4ac3-9860-f707a6673e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:42.025487 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060d476-cfad-4ac3-9860-f707a6673e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:42.027784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060d476-cfad-4ac3-9860-f707a6673e6b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:42.050188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 403/806] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:42.050188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060d476-cfad-4ac3-9860-f707a6673e6b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:42.068149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d8e38-53af-4b28-9602-f4143505c906 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:42.109263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c5b48-dc66-4092-a3dd-16d4d70e9991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:42.110009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c5b48-dc66-4092-a3dd-16d4d70e9991 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:42.110290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c5b48-dc66-4092-a3dd-16d4d70e9991 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:42.110565 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c5b48-dc66-4092-a3dd-16d4d70e9991 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:42.111277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c5b48-dc66-4092-a3dd-16d4d70e9991 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:42.111583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c5b48-dc66-4092-a3dd-16d4d70e9991 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:42.112983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c5b48-dc66-4092-a3dd-16d4d70e9991 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:42.118787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 404/807] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Jun 7 08:22:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:42.119818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c5b48-dc66-4092-a3dd-16d4d70e9991 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:42.185122 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060d476-cfad-4ac3-9860-f707a6673e6b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:42.208419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:42.234005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c5b48-dc66-4092-a3dd-16d4d70e9991 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:42.235216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d47065-0037-4306-a0d1-1a81ad3e22e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:42.236124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d47065-0037-4306-a0d1-1a81ad3e22e2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:42.236423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d47065-0037-4306-a0d1-1a81ad3e22e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:42.236701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d47065-0037-4306-a0d1-1a81ad3e22e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:42.263249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:42.274892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571e5c5-fe7c-4aaf-b41e-2477bb43eb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:42.275577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571e5c5-fe7c-4aaf-b41e-2477bb43eb12 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:42.275862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571e5c5-fe7c-4aaf-b41e-2477bb43eb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:42.276141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571e5c5-fe7c-4aaf-b41e-2477bb43eb12 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:42.277403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571e5c5-fe7c-4aaf-b41e-2477bb43eb12 None admin] RBAC: Adding query filter params (name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:42.277545 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571e5c5-fe7c-4aaf-b41e-2477bb43eb12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:42.279182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571e5c5-fe7c-4aaf-b41e-2477bb43eb12 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:42.288247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 405/808] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:42.289566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239412e-a20b-4915-9ed9-a75800862285 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:42.289859 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239412e-a20b-4915-9ed9-a75800862285 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:42.290140 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239412e-a20b-4915-9ed9-a75800862285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:42.311275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-8239412e-a20b-4915-9ed9-a75800862285 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:42.312919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8239412e-a20b-4915-9ed9-a75800862285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:42.324305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d47065-0037-4306-a0d1-1a81ad3e22e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:42.325792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d47065-0037-4306-a0d1-1a81ad3e22e2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:42.392884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 406/809] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:42.393785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239412e-a20b-4915-9ed9-a75800862285 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:42.460614 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 404/810] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:22:42.461476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d47065-0037-4306-a0d1-1a81ad3e22e2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:42.479238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:42.517616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:42.517902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239412e-a20b-4915-9ed9-a75800862285 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:42.561730 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8ff92-6671-4690-90bf-c5c3ea86c680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:42.562656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8ff92-6671-4690-90bf-c5c3ea86c680 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:42.562977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8ff92-6671-4690-90bf-c5c3ea86c680 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:42.563258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8ff92-6671-4690-90bf-c5c3ea86c680 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:42.571212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d8ff92-6671-4690-90bf-c5c3ea86c680 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:42.572701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d8ff92-6671-4690-90bf-c5c3ea86c680 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:42.575296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d47065-0037-4306-a0d1-1a81ad3e22e2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-86d8ff92-6671-4690-90bf-c5c3ea86c680 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:42.588347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:42.590215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:42.591845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:42.591845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:42.591845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:42.591845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:42.591845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:42.591845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:42.591845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:42.591845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:42.591845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 407/811] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:42.591845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8ff92-6671-4690-90bf-c5c3ea86c680 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:42.621642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b01d2-2e5d-407d-b16e-8c939800a7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:42.622322 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b01d2-2e5d-407d-b16e-8c939800a7e6 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:42.622608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b01d2-2e5d-407d-b16e-8c939800a7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:42.622882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b01d2-2e5d-407d-b16e-8c939800a7e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:42.623636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b01d2-2e5d-407d-b16e-8c939800a7e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:42.625012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b01d2-2e5d-407d-b16e-8c939800a7e6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:42.653522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 405/812] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jun 7 08:22:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:22:42.655424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b01d2-2e5d-407d-b16e-8c939800a7e6 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:42.718123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8ff92-6671-4690-90bf-c5c3ea86c680 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:42.759268 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:42.779106 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb61351-f842-4277-b4be-226ae162f35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:42.780429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb61351-f842-4277-b4be-226ae162f35b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:42.780891 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb61351-f842-4277-b4be-226ae162f35b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:42.781373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb61351-f842-4277-b4be-226ae162f35b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:42.802599 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb61351-f842-4277-b4be-226ae162f35b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:42.805325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb61351-f842-4277-b4be-226ae162f35b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:42.823652 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b01d2-2e5d-407d-b16e-8c939800a7e6 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-5cb61351-f842-4277-b4be-226ae162f35b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:42.833484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:42.835192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:42.839658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:42.839658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:42.839658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:42.839658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:42.839658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 408/813] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:42.839658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb61351-f842-4277-b4be-226ae162f35b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:42.841429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:42.909736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ff166-48f9-4ab4-a71b-796a98dd735e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:42.910565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ff166-48f9-4ab4-a71b-796a98dd735e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:42.910841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ff166-48f9-4ab4-a71b-796a98dd735e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:42.911126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ff166-48f9-4ab4-a71b-796a98dd735e None admin] PATH_INFO: `/v3/services/network` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:42.911791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ff166-48f9-4ab4-a71b-796a98dd735e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:42.913206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ff166-48f9-4ab4-a71b-796a98dd735e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-ed0ff166-48f9-4ab4-a71b-796a98dd735e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:42.927007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:42.929227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:42.938861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 406/814] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:42 2022] GET /identity/v3/services/network => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:42.941681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ff166-48f9-4ab4-a71b-796a98dd735e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:42.981068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb61351-f842-4277-b4be-226ae162f35b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:43.028655 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b3576-4db7-4606-8810-b04a096e0ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:43.029486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b3576-4db7-4606-8810-b04a096e0ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:43.029766 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b3576-4db7-4606-8810-b04a096e0ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:43.030047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b3576-4db7-4606-8810-b04a096e0ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:43.044052 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b3576-4db7-4606-8810-b04a096e0ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:43.045564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b3576-4db7-4606-8810-b04a096e0ef5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:43.064021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ff166-48f9-4ab4-a71b-796a98dd735e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-898b3576-4db7-4606-8810-b04a096e0ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:43.066908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:43.067841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:43.068728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:43.068728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:43.068728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:43.068728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:43.068728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 409/815] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:43.069319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b3576-4db7-4606-8810-b04a096e0ef5 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:43.111377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f29167-9e34-437c-a2c8-8b48a66a3b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:43.112131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f29167-9e34-437c-a2c8-8b48a66a3b1f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:43.112420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f29167-9e34-437c-a2c8-8b48a66a3b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:43.112698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f29167-9e34-437c-a2c8-8b48a66a3b1f None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:43.113626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f29167-9e34-437c-a2c8-8b48a66a3b1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:43.113836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f29167-9e34-437c-a2c8-8b48a66a3b1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:43.115343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f29167-9e34-437c-a2c8-8b48a66a3b1f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:43.119173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:43.132693 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 407/816] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:43.135137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f29167-9e34-437c-a2c8-8b48a66a3b1f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:43.172328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:43.240777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b3576-4db7-4606-8810-b04a096e0ef5 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:43.270624 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f29167-9e34-437c-a2c8-8b48a66a3b1f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:43.290060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffbd2a-d266-45d9-8a88-62170ed94abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:43.291145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffbd2a-d266-45d9-8a88-62170ed94abe None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:43.291683 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffbd2a-d266-45d9-8a88-62170ed94abe None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:43.292757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffbd2a-d266-45d9-8a88-62170ed94abe None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:43.294538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffbd2a-d266-45d9-8a88-62170ed94abe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:43.295369 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffbd2a-d266-45d9-8a88-62170ed94abe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:43.297908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffbd2a-d266-45d9-8a88-62170ed94abe None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:43.317403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea6fba-95ca-4ffe-9acb-773925e6bba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:43.318090 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea6fba-95ca-4ffe-9acb-773925e6bba2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:43.318381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea6fba-95ca-4ffe-9acb-773925e6bba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:43.319024 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea6fba-95ca-4ffe-9acb-773925e6bba2 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:43.319738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea6fba-95ca-4ffe-9acb-773925e6bba2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:43.320050 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea6fba-95ca-4ffe-9acb-773925e6bba2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:43.321405 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 410/817] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:43.322266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffbd2a-d266-45d9-8a88-62170ed94abe None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:43.326031 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea6fba-95ca-4ffe-9acb-773925e6bba2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:43.340212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 408/818] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jun 7 08:22:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:43.340212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea6fba-95ca-4ffe-9acb-773925e6bba2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:43.442718 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:43.458061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffbd2a-d266-45d9-8a88-62170ed94abe None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:43.482228 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:43.499312 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea6fba-95ca-4ffe-9acb-773925e6bba2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:43.500879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed97a0a-99f2-4e74-86b6-2ded0f6d017f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:43.501606 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed97a0a-99f2-4e74-86b6-2ded0f6d017f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:43.501885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed97a0a-99f2-4e74-86b6-2ded0f6d017f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:43.502372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed97a0a-99f2-4e74-86b6-2ded0f6d017f None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:43.503112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed97a0a-99f2-4e74-86b6-2ded0f6d017f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:43.504332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed97a0a-99f2-4e74-86b6-2ded0f6d017f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:43.505847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed97a0a-99f2-4e74-86b6-2ded0f6d017f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:43.522414 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 411/819] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:43.523800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed97a0a-99f2-4e74-86b6-2ded0f6d017f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:43.555613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde74c00-038f-417f-afe7-94e0c6a44e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:43.556534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde74c00-038f-417f-afe7-94e0c6a44e07 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:43.556824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde74c00-038f-417f-afe7-94e0c6a44e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:43.558122 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde74c00-038f-417f-afe7-94e0c6a44e07 None admin] PATH_INFO: `/v3/projects/07b861f95bf3498d990785ba72474d82/users/586ddffa77a54521b42b2aa50f16cb92/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:43.559959 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde74c00-038f-417f-afe7-94e0c6a44e07 None admin] RBAC: Authorizing `identity:create_grant(project_id=07b861f95bf3498d990785ba72474d82, user_id=586ddffa77a54521b42b2aa50f16cb92, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:43.561470 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde74c00-038f-417f-afe7-94e0c6a44e07 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:43.579332 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 409/820] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:22:43 2022] PUT /identity/v3/projects/07b861f95bf3498d990785ba72474d82/users/586ddffa77a54521b42b2aa50f16cb92/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:22:43.580064 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde74c00-038f-417f-afe7-94e0c6a44e07 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:43.720699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde74c00-038f-417f-afe7-94e0c6a44e07 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:43.749297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed97a0a-99f2-4e74-86b6-2ded0f6d017f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:43.769567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57351c-fb70-4ec7-b076-abe4f40ae190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:43.770268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57351c-fb70-4ec7-b076-abe4f40ae190 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:43.770553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57351c-fb70-4ec7-b076-abe4f40ae190 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:43.770824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57351c-fb70-4ec7-b076-abe4f40ae190 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:43.771570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57351c-fb70-4ec7-b076-abe4f40ae190 None admin] RBAC: Adding query filter params (type=network) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:43.771877 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57351c-fb70-4ec7-b076-abe4f40ae190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:43.773693 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57351c-fb70-4ec7-b076-abe4f40ae190 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:43.784832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 410/821] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jun 7 08:22:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:43.785895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57351c-fb70-4ec7-b076-abe4f40ae190 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:43.820462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:43.820663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2258784-b685-4cfd-8258-9855b5edc154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:43.821810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2258784-b685-4cfd-8258-9855b5edc154 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:43.822109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2258784-b685-4cfd-8258-9855b5edc154 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:43.822485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2258784-b685-4cfd-8258-9855b5edc154 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:43.836356 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2258784-b685-4cfd-8258-9855b5edc154 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:43.841235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2258784-b685-4cfd-8258-9855b5edc154 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-f2258784-b685-4cfd-8258-9855b5edc154 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:43.860061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:43.861345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:43.862297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:43.862297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:43.862297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:43.862297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:43.862297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:43.862297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:43.862297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:43.862297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:43.862297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:22:43.862297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:43.872813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 412/822] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jun 7 08:22:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:43.874967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2258784-b685-4cfd-8258-9855b5edc154 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:43.912061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:43.959149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57351c-fb70-4ec7-b076-abe4f40ae190 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:44.021223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3826317-3a0e-4cee-9f2f-205875255607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:44.022549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3826317-3a0e-4cee-9f2f-205875255607 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:44.023096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3826317-3a0e-4cee-9f2f-205875255607 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:44.023635 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3826317-3a0e-4cee-9f2f-205875255607 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:44.036396 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3826317-3a0e-4cee-9f2f-205875255607 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:44.038881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3826317-3a0e-4cee-9f2f-205875255607 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-d3826317-3a0e-4cee-9f2f-205875255607 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:44.063826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:44.065578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:44.068004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:44.068004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:44.068004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:44.068004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:44.068004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:44.068004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:44.068004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:44.068004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:44.068004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 411/823] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:44.068004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3826317-3a0e-4cee-9f2f-205875255607 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:44.094743 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2258784-b685-4cfd-8258-9855b5edc154 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:44.171121 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b49a64-322c-4360-807a-b2262005dda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:44.173041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b49a64-322c-4360-807a-b2262005dda3 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:44.173816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b49a64-322c-4360-807a-b2262005dda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:44.174901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b49a64-322c-4360-807a-b2262005dda3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:44.177567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b49a64-322c-4360-807a-b2262005dda3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:44.181035 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b49a64-322c-4360-807a-b2262005dda3 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:44.211280 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:44.229294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 413/824] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jun 7 08:22:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:22:44.231904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b49a64-322c-4360-807a-b2262005dda3 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:44.269812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3826317-3a0e-4cee-9f2f-205875255607 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:44.404431 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe0cd7-d853-461c-b8e2-d8e7e0df55d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:44.405577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0cd7-d853-461c-b8e2-d8e7e0df55d5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:44.406044 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0cd7-d853-461c-b8e2-d8e7e0df55d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:44.408920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0cd7-d853-461c-b8e2-d8e7e0df55d5 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:44.410662 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe0cd7-d853-461c-b8e2-d8e7e0df55d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:44.411183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe0cd7-d853-461c-b8e2-d8e7e0df55d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:44.413787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe0cd7-d853-461c-b8e2-d8e7e0df55d5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:44.466747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 412/825] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Jun 7 08:22:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:44.474629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe0cd7-d853-461c-b8e2-d8e7e0df55d5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:44.483031 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:44.596416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b49a64-322c-4360-807a-b2262005dda3 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:44.672541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15475d-4930-4a5f-be94-62d03f33bdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:44.674221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15475d-4930-4a5f-be94-62d03f33bdf1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:44.674668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15475d-4930-4a5f-be94-62d03f33bdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:44.675098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15475d-4930-4a5f-be94-62d03f33bdf1 None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:44.679224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15475d-4930-4a5f-be94-62d03f33bdf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:44.679224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15475d-4930-4a5f-be94-62d03f33bdf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:44.684378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15475d-4930-4a5f-be94-62d03f33bdf1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:44.731467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 414/826] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 498 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:44.737451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15475d-4930-4a5f-be94-62d03f33bdf1 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:44.759338 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe0cd7-d853-461c-b8e2-d8e7e0df55d5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:44.823179 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:44.832197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f783500-5360-46b0-be16-220322db2434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:44.833016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f783500-5360-46b0-be16-220322db2434 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:44.833369 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f783500-5360-46b0-be16-220322db2434 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:44.833638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f783500-5360-46b0-be16-220322db2434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:44.847818 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f783500-5360-46b0-be16-220322db2434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:44.849320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f783500-5360-46b0-be16-220322db2434 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-7f783500-5360-46b0-be16-220322db2434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:44.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:44.875020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:44.876734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:44.876734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:44.876734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:44.876734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:44.885595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 413/827] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:44.906184 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f783500-5360-46b0-be16-220322db2434 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:44.942592 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15475d-4930-4a5f-be94-62d03f33bdf1 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:44.975187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:44.986630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f7784-28cf-439e-a006-1de279d5c5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:44.987305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f7784-28cf-439e-a006-1de279d5c5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:44.987594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f7784-28cf-439e-a006-1de279d5c5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:44.987873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f7784-28cf-439e-a006-1de279d5c5d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:44.988730 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f7784-28cf-439e-a006-1de279d5c5d1 None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=3f2a2bf913d1412f996acf4401ba7673) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:44.989043 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f7784-28cf-439e-a006-1de279d5c5d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:44.990553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f7784-28cf-439e-a006-1de279d5c5d1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:44.998497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 415/828] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:44 2022] GET /identity/v3/role_assignments?user.id=3f2a2bf913d1412f996acf4401ba7673&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:45.092537 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f783500-5360-46b0-be16-220322db2434 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:45.136521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e0f5e-9fcd-4a2c-8281-019a44edb56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:45.137512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e0f5e-9fcd-4a2c-8281-019a44edb56e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:45.137981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e0f5e-9fcd-4a2c-8281-019a44edb56e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:45.138406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e0f5e-9fcd-4a2c-8281-019a44edb56e None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:45.139308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e0f5e-9fcd-4a2c-8281-019a44edb56e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:45.139771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e0f5e-9fcd-4a2c-8281-019a44edb56e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:45.141430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e0f5e-9fcd-4a2c-8281-019a44edb56e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:45.161670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 414/829] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:45.166284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f7784-28cf-439e-a006-1de279d5c5d1 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:45.195041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245164dc-ebf5-4360-89a1-e2b00d34b6f9 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:45.195269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245164dc-ebf5-4360-89a1-e2b00d34b6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:45.195552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245164dc-ebf5-4360-89a1-e2b00d34b6f9 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:45.197502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 415/830] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:22:45.205690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d16bf-846c-43cf-bd95-efeaba435e4e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:45.206114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d16bf-846c-43cf-bd95-efeaba435e4e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:45.206225 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d16bf-846c-43cf-bd95-efeaba435e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:45.230995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0e9d16bf-846c-43cf-bd95-efeaba435e4e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:45.232878 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0e9d16bf-846c-43cf-bd95-efeaba435e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:45.278563 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:45.318980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f7784-28cf-439e-a006-1de279d5c5d1 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:45.333132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 416/831] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:22:45.352199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d16bf-846c-43cf-bd95-efeaba435e4e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:45.366450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50fb55-e1a5-4ca3-9ccf-19b7c4c12c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:45.367261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50fb55-e1a5-4ca3-9ccf-19b7c4c12c06 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:45.367550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50fb55-e1a5-4ca3-9ccf-19b7c4c12c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:45.367830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50fb55-e1a5-4ca3-9ccf-19b7c4c12c06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:45.375771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50fb55-e1a5-4ca3-9ccf-19b7c4c12c06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:45.377279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50fb55-e1a5-4ca3-9ccf-19b7c4c12c06 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-cb50fb55-e1a5-4ca3-9ccf-19b7c4c12c06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:45.393922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:45.394877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:45.396276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:45.396276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:45.396276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:45.396276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:45.396276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:45.396276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:45.396276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:45.396276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:45.396276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 416/832] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:45.402270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50fb55-e1a5-4ca3-9ccf-19b7c4c12c06 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:45.487240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d16bf-846c-43cf-bd95-efeaba435e4e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:45.494903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:45.534454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f482f95-7e19-46fd-a951-027b1e7ee564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:45.535714 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f482f95-7e19-46fd-a951-027b1e7ee564 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:45.537943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f482f95-7e19-46fd-a951-027b1e7ee564 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:45.538652 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f482f95-7e19-46fd-a951-027b1e7ee564 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:45.539546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f482f95-7e19-46fd-a951-027b1e7ee564 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:45.542187 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f482f95-7e19-46fd-a951-027b1e7ee564 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-7f482f95-7e19-46fd-a951-027b1e7ee564 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:45.560687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:45.561791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:22:45.562638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:45.565545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 417/833] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:22:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:45.574558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f482f95-7e19-46fd-a951-027b1e7ee564 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:45.605113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50fb55-e1a5-4ca3-9ccf-19b7c4c12c06 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:45.642233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:45.686318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73451794-533e-414e-9a98-2a952f7f5141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:45.687000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73451794-533e-414e-9a98-2a952f7f5141 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:45.687282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73451794-533e-414e-9a98-2a952f7f5141 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:45.687570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73451794-533e-414e-9a98-2a952f7f5141 None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:45.688606 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73451794-533e-414e-9a98-2a952f7f5141 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:45.688768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73451794-533e-414e-9a98-2a952f7f5141 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:45.690789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73451794-533e-414e-9a98-2a952f7f5141 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:45.713161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 417/834] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:45.714728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73451794-533e-414e-9a98-2a952f7f5141 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:45.797439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f482f95-7e19-46fd-a951-027b1e7ee564 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:45.891186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83781fe6-7afe-472c-8dee-0a1d93f3712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:45.893056 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83781fe6-7afe-472c-8dee-0a1d93f3712a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:45.893830 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83781fe6-7afe-472c-8dee-0a1d93f3712a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:45.894731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83781fe6-7afe-472c-8dee-0a1d93f3712a None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:45.896679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83781fe6-7afe-472c-8dee-0a1d93f3712a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:45.897576 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83781fe6-7afe-472c-8dee-0a1d93f3712a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:45.902852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83781fe6-7afe-472c-8dee-0a1d93f3712a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:45.914496 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 418/835] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:22:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:45.921239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83781fe6-7afe-472c-8dee-0a1d93f3712a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:45.934062 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73451794-533e-414e-9a98-2a952f7f5141 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:45.950230 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:45.991106 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac14efd-3a8c-44ef-ad9a-eef737dd4fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:45.992371 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac14efd-3a8c-44ef-ad9a-eef737dd4fcb None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:45.992742 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac14efd-3a8c-44ef-ad9a-eef737dd4fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:45.993172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac14efd-3a8c-44ef-ad9a-eef737dd4fcb None admin] PATH_INFO: `/v3/projects/07b861f95bf3498d990785ba72474d82/users/26d6eb06b23b4db88261ae58d650736d/roles/9f07c51b3716428b89a49a756664305a` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:46.022169 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac14efd-3a8c-44ef-ad9a-eef737dd4fcb None admin] RBAC: Authorizing `identity:create_grant(project_id=07b861f95bf3498d990785ba72474d82, user_id=26d6eb06b23b4db88261ae58d650736d, role_id=9f07c51b3716428b89a49a756664305a)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:46.023113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac14efd-3a8c-44ef-ad9a-eef737dd4fcb None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:46.050985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 418/836] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:22:45 2022] PUT /identity/v3/projects/07b861f95bf3498d990785ba72474d82/users/26d6eb06b23b4db88261ae58d650736d/roles/9f07c51b3716428b89a49a756664305a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:22:46.171155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:46.179963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83781fe6-7afe-472c-8dee-0a1d93f3712a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:46.254771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39816039-ce87-432f-abdf-9c02ee646123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:46.256527 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39816039-ce87-432f-abdf-9c02ee646123 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:46.257420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39816039-ce87-432f-abdf-9c02ee646123 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:46.258227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39816039-ce87-432f-abdf-9c02ee646123 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:46.260074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39816039-ce87-432f-abdf-9c02ee646123 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:46.261031 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39816039-ce87-432f-abdf-9c02ee646123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:46.264185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39816039-ce87-432f-abdf-9c02ee646123 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:46.274748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 419/837] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:22:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:46.374509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92a70b-31b0-4259-925f-005df1886ef4 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:46.374788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92a70b-31b0-4259-925f-005df1886ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:46.375129 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92a70b-31b0-4259-925f-005df1886ef4 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:46.376873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 419/838] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:46.384288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16cda1-8762-4224-8ae5-f35026144279 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:46.384784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16cda1-8762-4224-8ae5-f35026144279 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:46.385302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16cda1-8762-4224-8ae5-f35026144279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:46.418531 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4f16cda1-8762-4224-8ae5-f35026144279 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:46.420883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4f16cda1-8762-4224-8ae5-f35026144279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:46.543318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 420/839] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:46.568344 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92a70b-31b0-4259-925f-005df1886ef4 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:46.736244 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92a70b-31b0-4259-925f-005df1886ef4 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:46.774846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9a16d-d626-435f-a021-da1ea55dc67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:46.776029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9a16d-d626-435f-a021-da1ea55dc67b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:46.777119 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9a16d-d626-435f-a021-da1ea55dc67b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:46.777653 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9a16d-d626-435f-a021-da1ea55dc67b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:46.790411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9a16d-d626-435f-a021-da1ea55dc67b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-4ab9a16d-d626-435f-a021-da1ea55dc67b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:46.799192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:46.800187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:46.801109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:46.801109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:46.801109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:46.801109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:46.801109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:46.801109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:46.801109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:46.801109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:46.801109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:46.801109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:46.801109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:46.801109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:46.804031 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 420/840] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:46.814709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16cda1-8762-4224-8ae5-f35026144279 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:46.918357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:46.925868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16cda1-8762-4224-8ae5-f35026144279 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:46.962534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6915e2-d575-406e-b778-64b0195ddcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:46.963212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6915e2-d575-406e-b778-64b0195ddcc9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:46.963497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6915e2-d575-406e-b778-64b0195ddcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:46.964448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6915e2-d575-406e-b778-64b0195ddcc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:46.965261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6915e2-d575-406e-b778-64b0195ddcc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:46.965566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6915e2-d575-406e-b778-64b0195ddcc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:46.971488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6915e2-d575-406e-b778-64b0195ddcc9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:46.981419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 421/841] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:46.986980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9a16d-d626-435f-a021-da1ea55dc67b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:47.012970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:47.104980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9a16d-d626-435f-a021-da1ea55dc67b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:47.143363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae17616-b2b7-4228-8a54-d93988c8cf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:47.144649 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae17616-b2b7-4228-8a54-d93988c8cf49 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:47.145161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae17616-b2b7-4228-8a54-d93988c8cf49 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:47.145637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae17616-b2b7-4228-8a54-d93988c8cf49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:47.157866 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae17616-b2b7-4228-8a54-d93988c8cf49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:47.159639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae17616-b2b7-4228-8a54-d93988c8cf49 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-2ae17616-b2b7-4228-8a54-d93988c8cf49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:47.178172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:47.179109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:47.180007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:47.180007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:47.180007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:47.180007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:47.182308 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 421/842] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:47.189490 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6915e2-d575-406e-b778-64b0195ddcc9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:47.331906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6915e2-d575-406e-b778-64b0195ddcc9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:47.367308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb26ae-acc9-47c8-b2c3-d0f29d7611c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:47.367982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb26ae-acc9-47c8-b2c3-d0f29d7611c8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:47.368270 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb26ae-acc9-47c8-b2c3-d0f29d7611c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:47.368549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb26ae-acc9-47c8-b2c3-d0f29d7611c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:47.369315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb26ae-acc9-47c8-b2c3-d0f29d7611c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:47.369630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb26ae-acc9-47c8-b2c3-d0f29d7611c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:47.371593 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb26ae-acc9-47c8-b2c3-d0f29d7611c8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:47.386803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 422/843] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:47.392746 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae17616-b2b7-4228-8a54-d93988c8cf49 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:47.482460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:47.523986 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae17616-b2b7-4228-8a54-d93988c8cf49 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:47.559001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c050a1-ac3f-4095-a564-f0eb6a2eee66 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:47.559211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c050a1-ac3f-4095-a564-f0eb6a2eee66 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:47.559462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c050a1-ac3f-4095-a564-f0eb6a2eee66 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:47.561062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 423/844] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:47.566870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a5796-236c-4a79-9fa8-2b65bcd9670a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:47.567234 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a5796-236c-4a79-9fa8-2b65bcd9670a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:47.567423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a5796-236c-4a79-9fa8-2b65bcd9670a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:47.579277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bc3eb-fd5a-4053-b10d-4100eae67e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:47.580708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bc3eb-fd5a-4053-b10d-4100eae67e8d None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:47.581269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bc3eb-fd5a-4053-b10d-4100eae67e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:47.581754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bc3eb-fd5a-4053-b10d-4100eae67e8d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:47.587181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6e6a5796-236c-4a79-9fa8-2b65bcd9670a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:47.588343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6e6a5796-236c-4a79-9fa8-2b65bcd9670a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:47.593881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67bc3eb-fd5a-4053-b10d-4100eae67e8d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:47.595868 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67bc3eb-fd5a-4053-b10d-4100eae67e8d None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:47.602185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-a67bc3eb-fd5a-4053-b10d-4100eae67e8d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:47.616008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:47.617033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:47.618475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:47.618475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:47.618475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:47.618475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:47.618475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:47.618475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:47.618475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:47.618475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:47.618475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:22:47.618475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:47.621661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 422/845] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:47.628671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bc3eb-fd5a-4053-b10d-4100eae67e8d None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:47.687716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 424/846] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:22:47.707996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a5796-236c-4a79-9fa8-2b65bcd9670a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:47.781801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bc3eb-fd5a-4053-b10d-4100eae67e8d None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:47.816120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:47.822974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3ad8e-1cc2-4f70-9d78-ef2617967c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:47.824828 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3ad8e-1cc2-4f70-9d78-ef2617967c8d None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:47.824948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a5796-236c-4a79-9fa8-2b65bcd9670a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:47.825838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3ad8e-1cc2-4f70-9d78-ef2617967c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:47.826328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3ad8e-1cc2-4f70-9d78-ef2617967c8d None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:47.827587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3ad8e-1cc2-4f70-9d78-ef2617967c8d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:47.828148 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3ad8e-1cc2-4f70-9d78-ef2617967c8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:47.830287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3ad8e-1cc2-4f70-9d78-ef2617967c8d None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:47.853122 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 423/847] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Jun 7 08:22:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:47.860450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3ad8e-1cc2-4f70-9d78-ef2617967c8d None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:47.889515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584998c0-b1c3-4cdd-bc2b-42d777c18941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:47.892728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584998c0-b1c3-4cdd-bc2b-42d777c18941 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:47.893356 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584998c0-b1c3-4cdd-bc2b-42d777c18941 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:47.893834 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584998c0-b1c3-4cdd-bc2b-42d777c18941 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:47.895792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584998c0-b1c3-4cdd-bc2b-42d777c18941 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:47.897749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584998c0-b1c3-4cdd-bc2b-42d777c18941 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-584998c0-b1c3-4cdd-bc2b-42d777c18941 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:47.923185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:47.924231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:22:47.925061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:47.932370 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 425/848] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Jun 7 08:22:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:47.935364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584998c0-b1c3-4cdd-bc2b-42d777c18941 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:47.951987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:48.027480 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3ad8e-1cc2-4f70-9d78-ef2617967c8d None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:48.083447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37f8f3-7132-47fb-a800-759968826828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:48.085162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37f8f3-7132-47fb-a800-759968826828 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:48.085728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37f8f3-7132-47fb-a800-759968826828 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:48.086067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584998c0-b1c3-4cdd-bc2b-42d777c18941 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:48.086570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37f8f3-7132-47fb-a800-759968826828 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:48.130198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62a6e0-4a38-4832-b201-610a30369f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:48.130619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62a6e0-4a38-4832-b201-610a30369f72 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:48.130864 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62a6e0-4a38-4832-b201-610a30369f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:48.131109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62a6e0-4a38-4832-b201-610a30369f72 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:48.131815 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62a6e0-4a38-4832-b201-610a30369f72 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:48.132123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62a6e0-4a38-4832-b201-610a30369f72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:48.133540 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62a6e0-4a38-4832-b201-610a30369f72 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:48.145124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 426/849] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jun 7 08:22:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:48.146346 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62a6e0-4a38-4832-b201-610a30369f72 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:48.191019 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:48.209299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37f8f3-7132-47fb-a800-759968826828 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:48.210836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37f8f3-7132-47fb-a800-759968826828 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:48.265621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:48.302412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62a6e0-4a38-4832-b201-610a30369f72 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:48.358938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 424/850] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:47 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:22:48.363263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69b88e-30ca-48e5-ace5-a452531de970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:48.364216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69b88e-30ca-48e5-ace5-a452531de970 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:48.364749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69b88e-30ca-48e5-ace5-a452531de970 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:48.365266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69b88e-30ca-48e5-ace5-a452531de970 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:48.366814 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37f8f3-7132-47fb-a800-759968826828 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:48.370519 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69b88e-30ca-48e5-ace5-a452531de970 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:48.371426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69b88e-30ca-48e5-ace5-a452531de970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:48.374377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69b88e-30ca-48e5-ace5-a452531de970 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:48.389682 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 427/851] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jun 7 08:22:48 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:48.476328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:48.497578 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37f8f3-7132-47fb-a800-759968826828 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:48.550910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ee2ae-d7fd-46bf-81d6-7eedb123fe3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:48.552009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ee2ae-d7fd-46bf-81d6-7eedb123fe3f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:48.552297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ee2ae-d7fd-46bf-81d6-7eedb123fe3f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:48.552574 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ee2ae-d7fd-46bf-81d6-7eedb123fe3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:48.567297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ee2ae-d7fd-46bf-81d6-7eedb123fe3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:48.568792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ee2ae-d7fd-46bf-81d6-7eedb123fe3f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-4d7ee2ae-d7fd-46bf-81d6-7eedb123fe3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:48.589589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:48.590589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:48.591613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:48.591613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:48.591613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:48.591613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:48.591613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 425/852] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:48.600328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69b88e-30ca-48e5-ace5-a452531de970 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:48.673820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab82ade-8839-44ff-8405-cc5307891439 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:48.674015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab82ade-8839-44ff-8405-cc5307891439 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:48.674244 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab82ade-8839-44ff-8405-cc5307891439 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:48.675936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 426/853] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:22:48.683974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8a849-c707-4ddf-bda7-9aee24332ad4 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:48.684259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8a849-c707-4ddf-bda7-9aee24332ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:48.684535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8a849-c707-4ddf-bda7-9aee24332ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:48.708105 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-6ef8a849-c707-4ddf-bda7-9aee24332ad4 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:48.709695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6ef8a849-c707-4ddf-bda7-9aee24332ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:48.751868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69b88e-30ca-48e5-ace5-a452531de970 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:48.804754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7427d9-8229-4247-86d0-3dc4444c24fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:48.806198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7427d9-8229-4247-86d0-3dc4444c24fb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:48.806608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7427d9-8229-4247-86d0-3dc4444c24fb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:48.806954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7427d9-8229-4247-86d0-3dc4444c24fb None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:48.808723 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7427d9-8229-4247-86d0-3dc4444c24fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:48.809103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7427d9-8229-4247-86d0-3dc4444c24fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:48.811151 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7427d9-8229-4247-86d0-3dc4444c24fb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:48.832382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 428/854] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:48.833809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca1c2d-d6f4-4524-9eba-28ad996fbc71 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:48.834101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca1c2d-d6f4-4524-9eba-28ad996fbc71 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:48.834381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca1c2d-d6f4-4524-9eba-28ad996fbc71 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:48.837392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:48.841189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 427/855] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:22:48.842285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8a849-c707-4ddf-bda7-9aee24332ad4 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:48.845318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 429/856] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:48 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:48.848515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74c172-65ae-4615-90b5-8d83c6e9c635 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:48.849053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74c172-65ae-4615-90b5-8d83c6e9c635 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:48.849814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74c172-65ae-4615-90b5-8d83c6e9c635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:48.872258 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0e74c172-65ae-4615-90b5-8d83c6e9c635 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:48.873969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0e74c172-65ae-4615-90b5-8d83c6e9c635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:48.929595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:48.968819 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8a849-c707-4ddf-bda7-9aee24332ad4 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:48.977442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 430/857] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:48.978223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74c172-65ae-4615-90b5-8d83c6e9c635 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:49.014404 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c08d2-5e2c-441b-b8fc-ab7ffb837134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:49.015205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c08d2-5e2c-441b-b8fc-ab7ffb837134 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:49.015488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c08d2-5e2c-441b-b8fc-ab7ffb837134 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:49.015761 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c08d2-5e2c-441b-b8fc-ab7ffb837134 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:49.024353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c08d2-5e2c-441b-b8fc-ab7ffb837134 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:49.026017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c08d2-5e2c-441b-b8fc-ab7ffb837134 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-d06c08d2-5e2c-441b-b8fc-ab7ffb837134 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:49.048731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:49.055770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:49.056770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:49.056770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:49.056770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:49.056770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:49.056770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:49.056770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:49.056770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:49.056770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:49.056770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 428/858] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:49.056770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c08d2-5e2c-441b-b8fc-ab7ffb837134 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:49.114165 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74c172-65ae-4615-90b5-8d83c6e9c635 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:49.145898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:49.155352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f28e03-8fbe-403f-9197-327815d57a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:49.156202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f28e03-8fbe-403f-9197-327815d57a8e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:49.156487 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f28e03-8fbe-403f-9197-327815d57a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:49.156761 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f28e03-8fbe-403f-9197-327815d57a8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:49.168584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f28e03-8fbe-403f-9197-327815d57a8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-62f28e03-8fbe-403f-9197-327815d57a8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:49.176354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:49.177642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 431/859] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:22:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:49.178705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f28e03-8fbe-403f-9197-327815d57a8e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:49.196807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c08d2-5e2c-441b-b8fc-ab7ffb837134 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:49.233364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:49.241710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668872c-365d-4fb0-ac8e-c34900c6040b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:49.241710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668872c-365d-4fb0-ac8e-c34900c6040b None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:49.243076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668872c-365d-4fb0-ac8e-c34900c6040b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:49.243076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668872c-365d-4fb0-ac8e-c34900c6040b None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:49.244020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668872c-365d-4fb0-ac8e-c34900c6040b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:49.246458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668872c-365d-4fb0-ac8e-c34900c6040b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:49.269548 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 429/860] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:22:49 2022] POST /identity/v3/services => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:22:49.271013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668872c-365d-4fb0-ac8e-c34900c6040b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:49.342999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f28e03-8fbe-403f-9197-327815d57a8e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:49.409642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484556eb-9de1-45a1-aa84-0295206c238d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:49.411098 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484556eb-9de1-45a1-aa84-0295206c238d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:49.412466 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484556eb-9de1-45a1-aa84-0295206c238d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:49.412757 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484556eb-9de1-45a1-aa84-0295206c238d None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:49.414931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484556eb-9de1-45a1-aa84-0295206c238d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:49.415366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484556eb-9de1-45a1-aa84-0295206c238d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:49.417293 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484556eb-9de1-45a1-aa84-0295206c238d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:49.437131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 432/861] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:49.444144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484556eb-9de1-45a1-aa84-0295206c238d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:49.461278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668872c-365d-4fb0-ac8e-c34900c6040b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:49.507589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ca45c-b49c-47b7-80cd-8a27a58568cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:49.508317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ca45c-b49c-47b7-80cd-8a27a58568cf None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:49.508616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ca45c-b49c-47b7-80cd-8a27a58568cf None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:49.508957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ca45c-b49c-47b7-80cd-8a27a58568cf None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:49.509946 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ca45c-b49c-47b7-80cd-8a27a58568cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:49.510256 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ca45c-b49c-47b7-80cd-8a27a58568cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:49.512903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ca45c-b49c-47b7-80cd-8a27a58568cf None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:49.520959 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:49.525541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 430/862] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:22:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:49.533032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ca45c-b49c-47b7-80cd-8a27a58568cf None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:49.612051 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:49.617518 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484556eb-9de1-45a1-aa84-0295206c238d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:49.669556 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ca45c-b49c-47b7-80cd-8a27a58568cf None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:49.676935 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d85c5b-922e-4c7f-b389-063f507ed400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:49.678209 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d85c5b-922e-4c7f-b389-063f507ed400 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:49.678684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d85c5b-922e-4c7f-b389-063f507ed400 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:49.679320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d85c5b-922e-4c7f-b389-063f507ed400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:49.680946 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d85c5b-922e-4c7f-b389-063f507ed400 None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=586ddffa77a54521b42b2aa50f16cb92) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:49.681515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d85c5b-922e-4c7f-b389-063f507ed400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:49.684260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d85c5b-922e-4c7f-b389-063f507ed400 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:49.699411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 433/863] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:49 2022] GET /identity/v3/role_assignments?user.id=586ddffa77a54521b42b2aa50f16cb92&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:49.727390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b88fc-c094-430f-8ade-56060a344c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:49.728659 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b88fc-c094-430f-8ade-56060a344c6b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:49.729200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b88fc-c094-430f-8ade-56060a344c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:49.730911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b88fc-c094-430f-8ade-56060a344c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:49.755020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b88fc-c094-430f-8ade-56060a344c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:49.757693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b88fc-c094-430f-8ade-56060a344c6b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-117b88fc-c094-430f-8ade-56060a344c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:49.791236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:49.794478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:49.796166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:49.796166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:49.796166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:49.796166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:49.796166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 431/864] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:49.805904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d85c5b-922e-4c7f-b389-063f507ed400 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:49.994811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d85c5b-922e-4c7f-b389-063f507ed400 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:50.014690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:50.062591 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc000cf7-4881-4cf4-bb4e-64837c5fbc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:50.064271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc000cf7-4881-4cf4-bb4e-64837c5fbc69 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:50.065030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc000cf7-4881-4cf4-bb4e-64837c5fbc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:50.065790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc000cf7-4881-4cf4-bb4e-64837c5fbc69 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:50.067560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc000cf7-4881-4cf4-bb4e-64837c5fbc69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:50.068510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc000cf7-4881-4cf4-bb4e-64837c5fbc69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:50.071439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc000cf7-4881-4cf4-bb4e-64837c5fbc69 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:50.097015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 434/865] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:50.102600 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b88fc-c094-430f-8ade-56060a344c6b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:50.254440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b88fc-c094-430f-8ade-56060a344c6b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:50.274510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:50.305818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e198e-6f0d-4c51-8149-118ac0e87cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:50.307167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e198e-6f0d-4c51-8149-118ac0e87cdd None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:50.308889 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e198e-6f0d-4c51-8149-118ac0e87cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:50.309467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e198e-6f0d-4c51-8149-118ac0e87cdd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:50.325430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e198e-6f0d-4c51-8149-118ac0e87cdd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:50.328233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e198e-6f0d-4c51-8149-118ac0e87cdd None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-222e198e-6f0d-4c51-8149-118ac0e87cdd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:50.358206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:50.359221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:50.360145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:50.360145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:50.360145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:50.360145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:50.360145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:50.360145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:50.360145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:50.360145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:50.360145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:22:50.360145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:50.361221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 432/866] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jun 7 08:22:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:50.366737 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc000cf7-4881-4cf4-bb4e-64837c5fbc69 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:50.481726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc000cf7-4881-4cf4-bb4e-64837c5fbc69 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:50.519807 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9e225-90c1-404e-acc3-7bd6bafcf94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:50.520417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9e225-90c1-404e-acc3-7bd6bafcf94d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:50.520703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9e225-90c1-404e-acc3-7bd6bafcf94d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:50.520982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9e225-90c1-404e-acc3-7bd6bafcf94d None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:50.521865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9e225-90c1-404e-acc3-7bd6bafcf94d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:50.522176 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9e225-90c1-404e-acc3-7bd6bafcf94d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:50.523610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9e225-90c1-404e-acc3-7bd6bafcf94d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:50.544032 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 435/867] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Jun 7 08:22:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:50.549948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e198e-6f0d-4c51-8149-118ac0e87cdd None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:50.678846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e198e-6f0d-4c51-8149-118ac0e87cdd None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:50.721389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516850ba-65e2-423a-b6d2-d1de5044a5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:50.722052 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516850ba-65e2-423a-b6d2-d1de5044a5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:50.722339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516850ba-65e2-423a-b6d2-d1de5044a5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:50.722622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516850ba-65e2-423a-b6d2-d1de5044a5b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:50.773036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:50.835010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516850ba-65e2-423a-b6d2-d1de5044a5b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:50.837851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516850ba-65e2-423a-b6d2-d1de5044a5b2 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:51.007143 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39128ae3-cb0f-471c-a9ba-934b9765c712 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:51.007668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39128ae3-cb0f-471c-a9ba-934b9765c712 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:51.007668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39128ae3-cb0f-471c-a9ba-934b9765c712 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:51.013324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 436/868] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:22:51.014663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5de410-666b-4506-b7b6-2cf1bdff100d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:51.014946 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5de410-666b-4506-b7b6-2cf1bdff100d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:51.015055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 433/869] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:50 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:22:51.015219 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5de410-666b-4506-b7b6-2cf1bdff100d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:51.020623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516850ba-65e2-423a-b6d2-d1de5044a5b2 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:51.040921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1e5de410-666b-4506-b7b6-2cf1bdff100d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:51.042241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1e5de410-666b-4506-b7b6-2cf1bdff100d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:51.067311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:51.109454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:51.145047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 437/870] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:51.150166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516850ba-65e2-423a-b6d2-d1de5044a5b2 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:51.169226 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5de410-666b-4506-b7b6-2cf1bdff100d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:51.202801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396b247-6935-48c5-ad8d-621c1af44c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:51.204812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396b247-6935-48c5-ad8d-621c1af44c93 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:51.205592 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396b247-6935-48c5-ad8d-621c1af44c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:51.206297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396b247-6935-48c5-ad8d-621c1af44c93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:51.227079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396b247-6935-48c5-ad8d-621c1af44c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:51.230423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396b247-6935-48c5-ad8d-621c1af44c93 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-e396b247-6935-48c5-ad8d-621c1af44c93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:51.268625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:51.269831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:51.270704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:51.270704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:51.270704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:51.270704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:51.277218 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 434/871] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:51.280776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396b247-6935-48c5-ad8d-621c1af44c93 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:51.325259 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5de410-666b-4506-b7b6-2cf1bdff100d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:51.379397 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e0191-5cf3-4852-949f-8eb77b1445cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:51.380118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e0191-5cf3-4852-949f-8eb77b1445cd None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:51.380454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e0191-5cf3-4852-949f-8eb77b1445cd None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:51.380653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e0191-5cf3-4852-949f-8eb77b1445cd None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:51.381449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e0191-5cf3-4852-949f-8eb77b1445cd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:51.382827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e0191-5cf3-4852-949f-8eb77b1445cd None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:51.401135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 438/872] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:22:51 2022] POST /identity/v3/services => generated 269 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:22:51.460880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396b247-6935-48c5-ad8d-621c1af44c93 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:51.504299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:51.551724 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a252500-7831-43f4-85d4-40dcabe7fc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:51.553120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a252500-7831-43f4-85d4-40dcabe7fc39 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:51.553776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a252500-7831-43f4-85d4-40dcabe7fc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:51.554393 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a252500-7831-43f4-85d4-40dcabe7fc39 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:51.555777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a252500-7831-43f4-85d4-40dcabe7fc39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:51.556491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a252500-7831-43f4-85d4-40dcabe7fc39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:51.559771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a252500-7831-43f4-85d4-40dcabe7fc39 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:51.585196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 435/873] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:51.590706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e0191-5cf3-4852-949f-8eb77b1445cd None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:51.614644 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:51.753891 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e0191-5cf3-4852-949f-8eb77b1445cd None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:51.795314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e1767-dd6d-4c6d-8b59-d465ffe820e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:51.796228 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e1767-dd6d-4c6d-8b59-d465ffe820e6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:51.796505 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e1767-dd6d-4c6d-8b59-d465ffe820e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:51.796833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e1767-dd6d-4c6d-8b59-d465ffe820e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:51.805717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e1767-dd6d-4c6d-8b59-d465ffe820e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:51.807200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e1767-dd6d-4c6d-8b59-d465ffe820e6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-5c6e1767-dd6d-4c6d-8b59-d465ffe820e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:51.824986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:51.826853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:51.828965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:51.828965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:51.828965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:51.828965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:51.828965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:51.828965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:51.828965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:51.828965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:51.828965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 439/874] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:51.832784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a252500-7831-43f4-85d4-40dcabe7fc39 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:51.966449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a252500-7831-43f4-85d4-40dcabe7fc39 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:51.992128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19937dcb-2a21-4012-8d31-c808bdcf11dc None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:51.992645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19937dcb-2a21-4012-8d31-c808bdcf11dc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:51.993235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19937dcb-2a21-4012-8d31-c808bdcf11dc None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:52.003350 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 440/875] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:52.005766 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f1fa6-679d-4081-bc14-cad26654529b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:52.006285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f1fa6-679d-4081-bc14-cad26654529b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:52.006796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f1fa6-679d-4081-bc14-cad26654529b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:52.024363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b3810-092d-44f7-b555-2e70e55ab8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:52.025533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b3810-092d-44f7-b555-2e70e55ab8cf None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:52.026073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b3810-092d-44f7-b555-2e70e55ab8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:52.026599 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b3810-092d-44f7-b555-2e70e55ab8cf None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:52.028242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b3810-092d-44f7-b555-2e70e55ab8cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:52.028784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b3810-092d-44f7-b555-2e70e55ab8cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:52.030608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-fe8f1fa6-679d-4081-bc14-cad26654529b None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:52.031586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b3810-092d-44f7-b555-2e70e55ab8cf None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:52.032015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fe8f1fa6-679d-4081-bc14-cad26654529b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:52.056026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 436/876] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:52.062022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b3810-092d-44f7-b555-2e70e55ab8cf None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:52.100239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:52.121329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 441/877] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:52.162213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f1fa6-679d-4081-bc14-cad26654529b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:52.174309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b3810-092d-44f7-b555-2e70e55ab8cf None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:52.181610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:52.216663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795d5f7-1d5f-4664-8eab-ca5a0a0f2eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:52.217425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795d5f7-1d5f-4664-8eab-ca5a0a0f2eaa None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:52.218119 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795d5f7-1d5f-4664-8eab-ca5a0a0f2eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:52.218469 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795d5f7-1d5f-4664-8eab-ca5a0a0f2eaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:52.219805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795d5f7-1d5f-4664-8eab-ca5a0a0f2eaa None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=26d6eb06b23b4db88261ae58d650736d) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:52.220209 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795d5f7-1d5f-4664-8eab-ca5a0a0f2eaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:52.221833 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795d5f7-1d5f-4664-8eab-ca5a0a0f2eaa None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:52.230227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 437/878] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:52 2022] GET /identity/v3/role_assignments?user.id=26d6eb06b23b4db88261ae58d650736d&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:52.332715 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f1fa6-679d-4081-bc14-cad26654529b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:52.373942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dde99c-ab70-462a-a3ff-9f2f5ca456da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:52.374856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dde99c-ab70-462a-a3ff-9f2f5ca456da None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:52.375138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dde99c-ab70-462a-a3ff-9f2f5ca456da None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:52.375438 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dde99c-ab70-462a-a3ff-9f2f5ca456da None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:52.376214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dde99c-ab70-462a-a3ff-9f2f5ca456da None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:52.377901 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dde99c-ab70-462a-a3ff-9f2f5ca456da None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f3dde99c-ab70-462a-a3ff-9f2f5ca456da None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:52.392501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:52.394371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:52.395851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 442/879] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:22:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:52.404796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795d5f7-1d5f-4664-8eab-ca5a0a0f2eaa None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:52.551941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:52.556982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795d5f7-1d5f-4664-8eab-ca5a0a0f2eaa None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:52.603844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8df34-5aac-4e33-990c-067f133b3f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:52.605239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8df34-5aac-4e33-990c-067f133b3f43 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:52.605591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8df34-5aac-4e33-990c-067f133b3f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:52.605854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8df34-5aac-4e33-990c-067f133b3f43 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:52.606569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8df34-5aac-4e33-990c-067f133b3f43 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:52.606891 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8df34-5aac-4e33-990c-067f133b3f43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:52.608422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8df34-5aac-4e33-990c-067f133b3f43 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:52.616503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 438/880] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:22:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:52.622084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dde99c-ab70-462a-a3ff-9f2f5ca456da None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:52.645885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3e5cd-1426-4bd1-93e8-02cfdc96c492 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:52.646185 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3e5cd-1426-4bd1-93e8-02cfdc96c492 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:52.646501 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3e5cd-1426-4bd1-93e8-02cfdc96c492 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:52.648074 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 439/881] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:22:52.654501 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edb575-1737-457b-b7fd-1b0c3edfbc8e None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:52.654872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edb575-1737-457b-b7fd-1b0c3edfbc8e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:52.655048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edb575-1737-457b-b7fd-1b0c3edfbc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:52.674982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-27edb575-1737-457b-b7fd-1b0c3edfbc8e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:52.676421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-27edb575-1737-457b-b7fd-1b0c3edfbc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:52.767897 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 440/882] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:52.779460 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:52.801136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edb575-1737-457b-b7fd-1b0c3edfbc8e None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:52.834741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dde99c-ab70-462a-a3ff-9f2f5ca456da None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:52.912638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0e21e-1f46-484d-8399-9877e8fc09dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:52.914273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0e21e-1f46-484d-8399-9877e8fc09dc None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:52.915045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0e21e-1f46-484d-8399-9877e8fc09dc None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:52.915918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0e21e-1f46-484d-8399-9877e8fc09dc None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:52.918004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0e21e-1f46-484d-8399-9877e8fc09dc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:52.918852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0e21e-1f46-484d-8399-9877e8fc09dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:52.921651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0e21e-1f46-484d-8399-9877e8fc09dc None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:52.931637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 443/883] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:22:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:52.939233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0e21e-1f46-484d-8399-9877e8fc09dc None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:52.969252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:52.988056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edb575-1737-457b-b7fd-1b0c3edfbc8e None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:53.078910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094bc717-7b47-4a34-8780-9fbab6308ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:53.079744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bc717-7b47-4a34-8780-9fbab6308ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:53.080024 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bc717-7b47-4a34-8780-9fbab6308ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:53.080289 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bc717-7b47-4a34-8780-9fbab6308ba5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:53.080964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094bc717-7b47-4a34-8780-9fbab6308ba5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:53.082414 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094bc717-7b47-4a34-8780-9fbab6308ba5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:53.096441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0e21e-1f46-484d-8399-9877e8fc09dc None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-094bc717-7b47-4a34-8780-9fbab6308ba5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:53.115538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:53.117330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:53.120749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 441/884] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jun 7 08:22:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:53.126351 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094bc717-7b47-4a34-8780-9fbab6308ba5 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:53.164710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dd70f-bc38-454a-bb85-26efea653542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:53.166979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dd70f-bc38-454a-bb85-26efea653542 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:53.167998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dd70f-bc38-454a-bb85-26efea653542 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:53.169269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dd70f-bc38-454a-bb85-26efea653542 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:53.171648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65dd70f-bc38-454a-bb85-26efea653542 None admin] RBAC: Adding query filter params (interface=public, service_id=be5900e6c1804eca921b19808f7a2dc8, region_id=RegionOne) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:53.173021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65dd70f-bc38-454a-bb85-26efea653542 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:53.177124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65dd70f-bc38-454a-bb85-26efea653542 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:53.181374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:53.192547 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 444/885] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jun 7 08:22:52 2022] GET /identity/v3/endpoints?service_id=be5900e6c1804eca921b19808f7a2dc8&interface=public®ion_id=RegionOne => generated 194 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:53.198762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dd70f-bc38-454a-bb85-26efea653542 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:53.277126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094bc717-7b47-4a34-8780-9fbab6308ba5 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:53.316037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f512d4-c3ec-419b-8311-c7cefa6ef022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:53.317009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f512d4-c3ec-419b-8311-c7cefa6ef022 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:53.317459 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f512d4-c3ec-419b-8311-c7cefa6ef022 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:53.317902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f512d4-c3ec-419b-8311-c7cefa6ef022 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:53.318789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f512d4-c3ec-419b-8311-c7cefa6ef022 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:53.319257 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f512d4-c3ec-419b-8311-c7cefa6ef022 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:53.320824 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f512d4-c3ec-419b-8311-c7cefa6ef022 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:53.326124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dd70f-bc38-454a-bb85-26efea653542 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:53.327832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 442/886] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jun 7 08:22:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:53.333629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f512d4-c3ec-419b-8311-c7cefa6ef022 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:53.360155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:53.378646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874e2a2-b36d-4909-b325-da02d06a2604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:53.379696 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874e2a2-b36d-4909-b325-da02d06a2604 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:53.380276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874e2a2-b36d-4909-b325-da02d06a2604 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:53.380770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874e2a2-b36d-4909-b325-da02d06a2604 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:53.381685 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874e2a2-b36d-4909-b325-da02d06a2604 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:53.383360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874e2a2-b36d-4909-b325-da02d06a2604 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:53.391682 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 445/887] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jun 7 08:22:53 2022] GET /identity/v3/services => generated 1268 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:22:53.451835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f512d4-c3ec-419b-8311-c7cefa6ef022 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:53.492586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fe335-58c9-451d-b926-eb229b065201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:53.493308 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fe335-58c9-451d-b926-eb229b065201 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:53.493596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fe335-58c9-451d-b926-eb229b065201 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:53.493870 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fe335-58c9-451d-b926-eb229b065201 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:53.494672 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fe335-58c9-451d-b926-eb229b065201 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:53.495118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fe335-58c9-451d-b926-eb229b065201 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:53.496805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fe335-58c9-451d-b926-eb229b065201 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:53.502745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 443/888] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jun 7 08:22:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:54.134433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405afea-963d-487f-bf1e-fc2010cb48f4 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:54.134957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405afea-963d-487f-bf1e-fc2010cb48f4 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:54.135070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405afea-963d-487f-bf1e-fc2010cb48f4 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:54.137633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 446/889] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:54.143382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f303f-7bd2-440e-b877-5ab51065f9bd None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:54.143675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f303f-7bd2-440e-b877-5ab51065f9bd None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:54.143962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f303f-7bd2-440e-b877-5ab51065f9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:54.162236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-9e2f303f-7bd2-440e-b877-5ab51065f9bd None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:54.163391 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9e2f303f-7bd2-440e-b877-5ab51065f9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:54.244805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 444/890] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:54.284511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405afea-963d-487f-bf1e-fc2010cb48f4 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:54.365440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:54.414117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405afea-963d-487f-bf1e-fc2010cb48f4 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:54.452442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a130a-8f43-42d5-be1e-92696c6a3a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:54.453819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a130a-8f43-42d5-be1e-92696c6a3a3a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:54.454257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a130a-8f43-42d5-be1e-92696c6a3a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:54.454684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a130a-8f43-42d5-be1e-92696c6a3a3a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:54.455524 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a130a-8f43-42d5-be1e-92696c6a3a3a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:54.457050 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a130a-8f43-42d5-be1e-92696c6a3a3a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-b63a130a-8f43-42d5-be1e-92696c6a3a3a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:54.472291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:54.473516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:22:54.474589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:54.477171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 447/891] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Jun 7 08:22:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:54.485102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f303f-7bd2-440e-b877-5ab51065f9bd None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:54.500031 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:54.598694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f303f-7bd2-440e-b877-5ab51065f9bd None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:54.642447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b60025-ecc9-41c5-bfae-7f58cd8a5bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:54.643146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b60025-ecc9-41c5-bfae-7f58cd8a5bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:54.643438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b60025-ecc9-41c5-bfae-7f58cd8a5bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:54.643715 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b60025-ecc9-41c5-bfae-7f58cd8a5bf0 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:54.644450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b60025-ecc9-41c5-bfae-7f58cd8a5bf0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:54.644756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b60025-ecc9-41c5-bfae-7f58cd8a5bf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:54.646225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b60025-ecc9-41c5-bfae-7f58cd8a5bf0 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:54.652285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 445/892] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jun 7 08:22:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:54.661741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a130a-8f43-42d5-be1e-92696c6a3a3a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:54.788772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a130a-8f43-42d5-be1e-92696c6a3a3a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:54.817457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:54.827535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda196c-445b-4194-ad2b-744701c97628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:54.828238 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda196c-445b-4194-ad2b-744701c97628 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:54.828872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda196c-445b-4194-ad2b-744701c97628 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:54.829198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda196c-445b-4194-ad2b-744701c97628 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:54.830173 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda196c-445b-4194-ad2b-744701c97628 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:54.830483 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda196c-445b-4194-ad2b-744701c97628 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:54.831995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda196c-445b-4194-ad2b-744701c97628 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:54.839040 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 448/893] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jun 7 08:22:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:54.845139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b60025-ecc9-41c5-bfae-7f58cd8a5bf0 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:54.965848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:54.999946 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b60025-ecc9-41c5-bfae-7f58cd8a5bf0 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:55.054644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8b95a-3673-428b-a747-78ebc18569fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:55.055237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8b95a-3673-428b-a747-78ebc18569fc None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:55.055525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8b95a-3673-428b-a747-78ebc18569fc None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:55.055804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8b95a-3673-428b-a747-78ebc18569fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:55.056602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8b95a-3673-428b-a747-78ebc18569fc None admin] RBAC: Adding query filter params (interface=public, service_id=ad9137efc3ff4628bd7a72cd57b56c6b, region_id=RegionOne) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:55.056917 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8b95a-3673-428b-a747-78ebc18569fc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:55.059197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8b95a-3673-428b-a747-78ebc18569fc None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:55.067978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 446/894] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jun 7 08:22:54 2022] GET /identity/v3/endpoints?service_id=ad9137efc3ff4628bd7a72cd57b56c6b&interface=public®ion_id=RegionOne => generated 194 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:55.074158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda196c-445b-4194-ad2b-744701c97628 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:55.108543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765efd72-a26d-44bd-b288-c8af2fda1d88 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:55.108858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765efd72-a26d-44bd-b288-c8af2fda1d88 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:55.109159 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765efd72-a26d-44bd-b288-c8af2fda1d88 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:55.110875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 447/895] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:55.116214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c182129-103d-4b9c-9e68-2bf1ef0eeddb None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:55.116508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c182129-103d-4b9c-9e68-2bf1ef0eeddb None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:55.116796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c182129-103d-4b9c-9e68-2bf1ef0eeddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:55.134954 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-6c182129-103d-4b9c-9e68-2bf1ef0eeddb None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:55.136995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6c182129-103d-4b9c-9e68-2bf1ef0eeddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:55.221874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda196c-445b-4194-ad2b-744701c97628 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:55.225188 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 448/896] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:22:55.265273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc08e5c-eb6a-41cf-9f3e-cda039209644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:55.267314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c182129-103d-4b9c-9e68-2bf1ef0eeddb None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:55.269894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc08e5c-eb6a-41cf-9f3e-cda039209644 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:55.270834 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc08e5c-eb6a-41cf-9f3e-cda039209644 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:55.271670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc08e5c-eb6a-41cf-9f3e-cda039209644 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:55.273392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc08e5c-eb6a-41cf-9f3e-cda039209644 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:55.277320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc08e5c-eb6a-41cf-9f3e-cda039209644 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:55.297842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 449/897] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jun 7 08:22:55 2022] GET /identity/v3/services => generated 1268 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:22:55.392576 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:55.438184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c182129-103d-4b9c-9e68-2bf1ef0eeddb None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:55.479534 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6804b8f-e3cb-4964-bcce-30b3189adeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:55.480389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6804b8f-e3cb-4964-bcce-30b3189adeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:55.480877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6804b8f-e3cb-4964-bcce-30b3189adeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:55.481215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6804b8f-e3cb-4964-bcce-30b3189adeb4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:55.495932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6804b8f-e3cb-4964-bcce-30b3189adeb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-b6804b8f-e3cb-4964-bcce-30b3189adeb4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:55.506510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:55.507513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:55.508340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:55.508340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:55.508340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:55.508340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:55.508340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:55.508340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:55.508340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:55.508340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:55.508340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:55.508340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:55.508340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:22:55.508340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:55.508968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 449/898] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Jun 7 08:22:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:55.516127 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc08e5c-eb6a-41cf-9f3e-cda039209644 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:55.610727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:55.684085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc08e5c-eb6a-41cf-9f3e-cda039209644 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:55.754376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45af69-a20f-4bad-83fc-dd6ae5d044cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:55.755246 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45af69-a20f-4bad-83fc-dd6ae5d044cd None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:55.755534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45af69-a20f-4bad-83fc-dd6ae5d044cd None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:55.755957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45af69-a20f-4bad-83fc-dd6ae5d044cd None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:55.757416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45af69-a20f-4bad-83fc-dd6ae5d044cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:55.757868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45af69-a20f-4bad-83fc-dd6ae5d044cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:55.759462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45af69-a20f-4bad-83fc-dd6ae5d044cd None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:55.775090 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 450/899] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jun 7 08:22:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:55.781421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6804b8f-e3cb-4964-bcce-30b3189adeb4 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:55.910535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6804b8f-e3cb-4964-bcce-30b3189adeb4 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:55.947595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:55.950714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7dc20-03c1-464a-8fc8-6a5286eee9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:55.951967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7dc20-03c1-464a-8fc8-6a5286eee9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:55.952465 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7dc20-03c1-464a-8fc8-6a5286eee9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:55.952927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7dc20-03c1-464a-8fc8-6a5286eee9b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:55.966959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7dc20-03c1-464a-8fc8-6a5286eee9b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:55.968537 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7dc20-03c1-464a-8fc8-6a5286eee9b1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-18a7dc20-03c1-464a-8fc8-6a5286eee9b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:55.988575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:55.989570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:55.990443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:55.990443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:55.990443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:55.990443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:55.990443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 450/900] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:55.998388 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45af69-a20f-4bad-83fc-dd6ae5d044cd None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:56.113984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45af69-a20f-4bad-83fc-dd6ae5d044cd None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:56.133462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:56.156371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c37417-ab7f-4393-ac44-610944bb0796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:56.157501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c37417-ab7f-4393-ac44-610944bb0796 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:56.157997 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c37417-ab7f-4393-ac44-610944bb0796 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:56.158477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c37417-ab7f-4393-ac44-610944bb0796 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:56.159523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c37417-ab7f-4393-ac44-610944bb0796 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:56.160126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c37417-ab7f-4393-ac44-610944bb0796 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:56.161992 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c37417-ab7f-4393-ac44-610944bb0796 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:56.177041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 451/901] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:56.184331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7dc20-03c1-464a-8fc8-6a5286eee9b1 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:56.341036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7dc20-03c1-464a-8fc8-6a5286eee9b1 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:56.375406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23816709-c781-4eab-b704-2d04f8731c6a None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:56.376226 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23816709-c781-4eab-b704-2d04f8731c6a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:56.376226 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23816709-c781-4eab-b704-2d04f8731c6a None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:56.377936 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 452/902] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:56.384107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3aef65-169a-48a0-a039-6e6dbaa4adcb None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:56.384526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3aef65-169a-48a0-a039-6e6dbaa4adcb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:56.384807 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3aef65-169a-48a0-a039-6e6dbaa4adcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:56.405450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1a3aef65-169a-48a0-a039-6e6dbaa4adcb None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:56.406881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1a3aef65-169a-48a0-a039-6e6dbaa4adcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:56.420726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387cd00-cd39-4785-98ed-5765b5da3764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:56.421664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387cd00-cd39-4785-98ed-5765b5da3764 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:56.421974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387cd00-cd39-4785-98ed-5765b5da3764 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:56.422278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387cd00-cd39-4785-98ed-5765b5da3764 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:56.432840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387cd00-cd39-4785-98ed-5765b5da3764 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:56.434490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387cd00-cd39-4785-98ed-5765b5da3764 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-c387cd00-cd39-4785-98ed-5765b5da3764 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:22:56.452505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:56.454018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:56.455193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:56.455193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:56.455193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:22:56.455193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:22:56.455193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:22:56.455193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:22:56.455193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:22:56.455193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:22:56.455193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:22:56.455193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:56.459219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 451/903] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jun 7 08:22:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:56.460920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387cd00-cd39-4785-98ed-5765b5da3764 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:56.526008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:56.534134 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 453/904] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:22:56.560272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3aef65-169a-48a0-a039-6e6dbaa4adcb None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:56.601123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387cd00-cd39-4785-98ed-5765b5da3764 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:56.643829 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae5823-cb9c-4680-90ea-ab6cc0cfb208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:56.644505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae5823-cb9c-4680-90ea-ab6cc0cfb208 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:56.644794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae5823-cb9c-4680-90ea-ab6cc0cfb208 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:56.645077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae5823-cb9c-4680-90ea-ab6cc0cfb208 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:56.645932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae5823-cb9c-4680-90ea-ab6cc0cfb208 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:56.646243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae5823-cb9c-4680-90ea-ab6cc0cfb208 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:56.647678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae5823-cb9c-4680-90ea-ab6cc0cfb208 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:56.672399 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 452/905] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Jun 7 08:22:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:56.674610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ea468-911d-45f4-b2ed-ba7ca94011e3 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:56.674923 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ea468-911d-45f4-b2ed-ba7ca94011e3 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:56.675210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ea468-911d-45f4-b2ed-ba7ca94011e3 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:56.677389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 453/906] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:56.677968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ea468-911d-45f4-b2ed-ba7ca94011e3 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:56.694351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3aef65-169a-48a0-a039-6e6dbaa4adcb None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:56.702768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:56.740313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ffbad-6970-464b-8b10-e5449d34c27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:56.741028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ffbad-6970-464b-8b10-e5449d34c27e None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:56.741372 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ffbad-6970-464b-8b10-e5449d34c27e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:56.741619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ffbad-6970-464b-8b10-e5449d34c27e None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:56.742360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ffbad-6970-464b-8b10-e5449d34c27e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:56.743733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ffbad-6970-464b-8b10-e5449d34c27e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:56.763185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 454/907] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:22:56 2022] POST /identity/v3/services => generated 241 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:22:56.764474 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbc58b-d48c-4aea-ac99-e3a103e18957 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:56.764775 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbc58b-d48c-4aea-ac99-e3a103e18957 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:56.765035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbc58b-d48c-4aea-ac99-e3a103e18957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:56.795553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7ebbc58b-d48c-4aea-ac99-e3a103e18957 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:56.798102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7ebbc58b-d48c-4aea-ac99-e3a103e18957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:56.877010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:56.893434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ea468-911d-45f4-b2ed-ba7ca94011e3 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:56.917925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 455/908] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:56.939575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ca0c6-fe31-4a04-9ec5-9244d450af36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:56.940444 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ca0c6-fe31-4a04-9ec5-9244d450af36 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:56.940856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ca0c6-fe31-4a04-9ec5-9244d450af36 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:56.941274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ca0c6-fe31-4a04-9ec5-9244d450af36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:56.944747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbc58b-d48c-4aea-ac99-e3a103e18957 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:57.020381 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ca0c6-fe31-4a04-9ec5-9244d450af36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:57.021901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ca0c6-fe31-4a04-9ec5-9244d450af36 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:57.103758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:57.122430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbc58b-d48c-4aea-ac99-e3a103e18957 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:57.168961 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:57.178215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 454/909] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:56 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:22:57.179134 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ca0c6-fe31-4a04-9ec5-9244d450af36 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:57.190542 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b635b-ca04-41e2-83ca-63f835fdacab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:57.192443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b635b-ca04-41e2-83ca-63f835fdacab None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:57.193206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b635b-ca04-41e2-83ca-63f835fdacab None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:57.194141 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b635b-ca04-41e2-83ca-63f835fdacab None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:57.195694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b635b-ca04-41e2-83ca-63f835fdacab None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:57.199543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b635b-ca04-41e2-83ca-63f835fdacab None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-545b635b-ca04-41e2-83ca-63f835fdacab None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:57.223607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:57.225285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:22:57.226827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:57.231789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 456/910] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:22:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:57.241491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b635b-ca04-41e2-83ca-63f835fdacab None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:57.295780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ca0c6-fe31-4a04-9ec5-9244d450af36 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:57.337315 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae4f79-5de0-4d19-af4b-b7e52008f903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:57.338114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae4f79-5de0-4d19-af4b-b7e52008f903 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:57.338477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae4f79-5de0-4d19-af4b-b7e52008f903 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:57.338926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae4f79-5de0-4d19-af4b-b7e52008f903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:57.352133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae4f79-5de0-4d19-af4b-b7e52008f903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:57.353875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae4f79-5de0-4d19-af4b-b7e52008f903 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-5fae4f79-5de0-4d19-af4b-b7e52008f903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:22:57.377141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:57.379022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:57.380831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:22:57.380831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:22:57.380831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:22:57.380831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:57.380831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 455/911] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:57.384008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae4f79-5de0-4d19-af4b-b7e52008f903 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:57.436540 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b635b-ca04-41e2-83ca-63f835fdacab None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:57.485455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b872a7-005c-4038-b814-3881ac986ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:57.486166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b872a7-005c-4038-b814-3881ac986ffe None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:57.486436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b872a7-005c-4038-b814-3881ac986ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:57.486720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b872a7-005c-4038-b814-3881ac986ffe None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:57.487419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b872a7-005c-4038-b814-3881ac986ffe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:57.487721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b872a7-005c-4038-b814-3881ac986ffe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:57.489118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b872a7-005c-4038-b814-3881ac986ffe None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:57.501131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 457/912] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:22:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:57.505777 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b872a7-005c-4038-b814-3881ac986ffe None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:57.508406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:57.569467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:57.579742 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae4f79-5de0-4d19-af4b-b7e52008f903 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:57.621428 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437b22a-8d18-4bdd-81f2-9cbb5758afd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:57.622339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437b22a-8d18-4bdd-81f2-9cbb5758afd2 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:57.622818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437b22a-8d18-4bdd-81f2-9cbb5758afd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:57.623092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437b22a-8d18-4bdd-81f2-9cbb5758afd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:57.623911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437b22a-8d18-4bdd-81f2-9cbb5758afd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:57.624637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437b22a-8d18-4bdd-81f2-9cbb5758afd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:57.625889 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437b22a-8d18-4bdd-81f2-9cbb5758afd2 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:57.644958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 456/913] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:57.651096 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437b22a-8d18-4bdd-81f2-9cbb5758afd2 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:57.681521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b872a7-005c-4038-b814-3881ac986ffe None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:57.728386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7b832-5d48-4cd6-9d42-8476bd816da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:57.730257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7b832-5d48-4cd6-9d42-8476bd816da4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:57.730747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7b832-5d48-4cd6-9d42-8476bd816da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:57.731361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7b832-5d48-4cd6-9d42-8476bd816da4 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:57.732435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7b832-5d48-4cd6-9d42-8476bd816da4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:57.733041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7b832-5d48-4cd6-9d42-8476bd816da4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:57.734999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7b832-5d48-4cd6-9d42-8476bd816da4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:57.743090 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 458/914] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jun 7 08:22:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:57.749342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7b832-5d48-4cd6-9d42-8476bd816da4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:57.794583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437b22a-8d18-4bdd-81f2-9cbb5758afd2 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:57.811587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:57.847996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15115c-f0b0-407b-8268-2b32aca753f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:57.849609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15115c-f0b0-407b-8268-2b32aca753f7 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:57.850500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15115c-f0b0-407b-8268-2b32aca753f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:57.850991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15115c-f0b0-407b-8268-2b32aca753f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:57.862195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:57.864009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15115c-f0b0-407b-8268-2b32aca753f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:57.866290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15115c-f0b0-407b-8268-2b32aca753f7 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:57.879295 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7b832-5d48-4cd6-9d42-8476bd816da4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-cf15115c-f0b0-407b-8268-2b32aca753f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:22:57.884412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:57.886204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:57.887807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:22:57.887807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:22:57.887807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:22:57.887807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:22:57.887807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:22:57.887807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:22:57.887807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:22:57.887807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:57.888400 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 457/915] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:22:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:57.898896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15115c-f0b0-407b-8268-2b32aca753f7 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:57.927746 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ac000-ca2e-4b0f-b5a9-214028cf5ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:57.928619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ac000-ca2e-4b0f-b5a9-214028cf5ab0 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:57.928903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ac000-ca2e-4b0f-b5a9-214028cf5ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:57.929223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ac000-ca2e-4b0f-b5a9-214028cf5ab0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:57.930479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ac000-ca2e-4b0f-b5a9-214028cf5ab0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:57.932421 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ac000-ca2e-4b0f-b5a9-214028cf5ab0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:57.957415 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 459/916] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jun 7 08:22:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:22:57.958819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53a6a8-344d-4d98-9acc-fcfe03e35436 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:57.959180 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53a6a8-344d-4d98-9acc-fcfe03e35436 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:57.959470 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53a6a8-344d-4d98-9acc-fcfe03e35436 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:57.965593 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 460/917] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:57.969167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f055091-bbac-4804-97ae-c041face515c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:57.969847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f055091-bbac-4804-97ae-c041face515c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:57.970332 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f055091-bbac-4804-97ae-c041face515c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:57.989792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0f055091-bbac-4804-97ae-c041face515c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:57.991304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0f055091-bbac-4804-97ae-c041face515c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:58.043588 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15115c-f0b0-407b-8268-2b32aca753f7 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:58.077478 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 461/918] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:58.098798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7051fb-d59c-4bbe-ad13-a18873ffbbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:58.099830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7051fb-d59c-4bbe-ad13-a18873ffbbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:58.100311 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7051fb-d59c-4bbe-ad13-a18873ffbbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:58.100761 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7051fb-d59c-4bbe-ad13-a18873ffbbf9 None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:58.101820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7051fb-d59c-4bbe-ad13-a18873ffbbf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:58.102391 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7051fb-d59c-4bbe-ad13-a18873ffbbf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:58.103665 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f055091-bbac-4804-97ae-c041face515c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:58.110005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7051fb-d59c-4bbe-ad13-a18873ffbbf9 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:58.133427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 458/919] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:22:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:58.139567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7051fb-d59c-4bbe-ad13-a18873ffbbf9 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:58.199455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:58.202767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:58.240154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f055091-bbac-4804-97ae-c041face515c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:58.294581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885891d1-6700-463f-add5-82615e8b14f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:58.295918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885891d1-6700-463f-add5-82615e8b14f1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:58.296390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885891d1-6700-463f-add5-82615e8b14f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:58.296851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885891d1-6700-463f-add5-82615e8b14f1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:58.297707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885891d1-6700-463f-add5-82615e8b14f1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:58.299575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885891d1-6700-463f-add5-82615e8b14f1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:58.315168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7051fb-d59c-4bbe-ad13-a18873ffbbf9 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-885891d1-6700-463f-add5-82615e8b14f1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:58.320052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:58.321982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:22:58.324408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 462/920] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Jun 7 08:22:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:22:58.331565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885891d1-6700-463f-add5-82615e8b14f1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:58.360618 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c2e9e-4344-4cb1-a17e-34d599852cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:58.361497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2e9e-4344-4cb1-a17e-34d599852cea None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:58.361804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2e9e-4344-4cb1-a17e-34d599852cea None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:58.362117 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2e9e-4344-4cb1-a17e-34d599852cea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:58.363552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c2e9e-4344-4cb1-a17e-34d599852cea None admin] RBAC: Adding query filter params (role.id=7db380ac539f4b79a45b254e5f57896d, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=26d6eb06b23b4db88261ae58d650736d) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:58.363896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c2e9e-4344-4cb1-a17e-34d599852cea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:58.365988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c2e9e-4344-4cb1-a17e-34d599852cea None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:58.376269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 459/921] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:58 2022] GET /identity/v3/role_assignments?user.id=26d6eb06b23b4db88261ae58d650736d&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=7db380ac539f4b79a45b254e5f57896d => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:58.464682 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885891d1-6700-463f-add5-82615e8b14f1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:58.511290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798fc82-201d-4e17-9d0c-c147dc45870b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:58.512025 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798fc82-201d-4e17-9d0c-c147dc45870b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:58.512414 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798fc82-201d-4e17-9d0c-c147dc45870b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:58.512797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798fc82-201d-4e17-9d0c-c147dc45870b None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:58.513744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798fc82-201d-4e17-9d0c-c147dc45870b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:58.514157 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798fc82-201d-4e17-9d0c-c147dc45870b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:58.515657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798fc82-201d-4e17-9d0c-c147dc45870b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:58.517038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:58.523132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 463/922] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jun 7 08:22:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:22:58.529628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c2e9e-4344-4cb1-a17e-34d599852cea None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:58.648516 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:58.661116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c2e9e-4344-4cb1-a17e-34d599852cea None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:58.714905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973db3b-fff7-4830-8480-0870cf40f10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:58.715992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973db3b-fff7-4830-8480-0870cf40f10a None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:58.716509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973db3b-fff7-4830-8480-0870cf40f10a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:58.716921 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973db3b-fff7-4830-8480-0870cf40f10a None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:58.718258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973db3b-fff7-4830-8480-0870cf40f10a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:58.718791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973db3b-fff7-4830-8480-0870cf40f10a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:58.721176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973db3b-fff7-4830-8480-0870cf40f10a None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:58.729758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 460/923] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jun 7 08:22:58 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:58.735105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798fc82-201d-4e17-9d0c-c147dc45870b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:58.866170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798fc82-201d-4e17-9d0c-c147dc45870b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:58.915903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93baca91-1116-492c-9f05-4f0f1f89ca1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:58.917341 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93baca91-1116-492c-9f05-4f0f1f89ca1d None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:58.917537 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93baca91-1116-492c-9f05-4f0f1f89ca1d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:58.917823 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93baca91-1116-492c-9f05-4f0f1f89ca1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:58.918599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93baca91-1116-492c-9f05-4f0f1f89ca1d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:58.919991 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93baca91-1116-492c-9f05-4f0f1f89ca1d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:58.942632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 464/924] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jun 7 08:22:58 2022] POST /identity/v3/endpoints => generated 347 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:22:59.451700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60921d57-0880-4b73-9907-1693eee00e03 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:59.452240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60921d57-0880-4b73-9907-1693eee00e03 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:59.452650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60921d57-0880-4b73-9907-1693eee00e03 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:59.454847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 461/925] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:22:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:59.460950 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449f2c6-0e68-4984-9589-f17d3a63d005 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:59.461315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449f2c6-0e68-4984-9589-f17d3a63d005 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:59.461547 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449f2c6-0e68-4984-9589-f17d3a63d005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:59.479954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9449f2c6-0e68-4984-9589-f17d3a63d005 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:59.481193 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9449f2c6-0e68-4984-9589-f17d3a63d005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:59.541105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:59.571670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 465/926] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:22:59.613194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60921d57-0880-4b73-9907-1693eee00e03 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:59.749917 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60921d57-0880-4b73-9907-1693eee00e03 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:59.794957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061bd06-bcc1-4132-8116-ba860455a6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:59.795779 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061bd06-bcc1-4132-8116-ba860455a6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:59.796060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061bd06-bcc1-4132-8116-ba860455a6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:59.796330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061bd06-bcc1-4132-8116-ba860455a6b7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:59.797008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061bd06-bcc1-4132-8116-ba860455a6b7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:59.798406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061bd06-bcc1-4132-8116-ba860455a6b7 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-6061bd06-bcc1-4132-8116-ba860455a6b7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:59.813438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:59.814496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:22:59.815334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 462/927] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jun 7 08:22:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:59.823002 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449f2c6-0e68-4984-9589-f17d3a63d005 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:59.961894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449f2c6-0e68-4984-9589-f17d3a63d005 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:59.998506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1d324-be53-48e5-824c-b4a6121c2e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:59.999182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1d324-be53-48e5-824c-b4a6121c2e43 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:59.999465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1d324-be53-48e5-824c-b4a6121c2e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:59.999737 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1d324-be53-48e5-824c-b4a6121c2e43 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:00.000503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1d324-be53-48e5-824c-b4a6121c2e43 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:00.000748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1d324-be53-48e5-824c-b4a6121c2e43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:00.002141 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1d324-be53-48e5-824c-b4a6121c2e43 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:00.007656 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 466/928] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jun 7 08:22:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:00.013603 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061bd06-bcc1-4132-8116-ba860455a6b7 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:00.040968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:00.125997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061bd06-bcc1-4132-8116-ba860455a6b7 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:00.170948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa3ee2-7c2f-474a-812d-d43ce49d8518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:00.171680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa3ee2-7c2f-474a-812d-d43ce49d8518 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:00.171963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa3ee2-7c2f-474a-812d-d43ce49d8518 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:00.172240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa3ee2-7c2f-474a-812d-d43ce49d8518 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:00.172983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa3ee2-7c2f-474a-812d-d43ce49d8518 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:00.173339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa3ee2-7c2f-474a-812d-d43ce49d8518 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:00.174788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa3ee2-7c2f-474a-812d-d43ce49d8518 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:00.181221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 463/929] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jun 7 08:23:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:00.187561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1d324-be53-48e5-824c-b4a6121c2e43 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:00.240529 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:00.321374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1d324-be53-48e5-824c-b4a6121c2e43 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:00.375695 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30834c0c-c839-4e64-9518-e069e1fe5570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:00.376484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30834c0c-c839-4e64-9518-e069e1fe5570 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:00.376771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30834c0c-c839-4e64-9518-e069e1fe5570 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:00.377063 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30834c0c-c839-4e64-9518-e069e1fe5570 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:00.378046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30834c0c-c839-4e64-9518-e069e1fe5570 None admin] RBAC: Adding query filter params (interface=public, service_id=98f2668e73c546d0a442150e4596095a, region_id=RegionOne) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:00.378375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30834c0c-c839-4e64-9518-e069e1fe5570 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:00.379817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30834c0c-c839-4e64-9518-e069e1fe5570 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:00.386886 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 467/930] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jun 7 08:23:00 2022] GET /identity/v3/endpoints?service_id=98f2668e73c546d0a442150e4596095a&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:00.392543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa3ee2-7c2f-474a-812d-d43ce49d8518 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:00.487341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:00.503909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa3ee2-7c2f-474a-812d-d43ce49d8518 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:00.508341 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89ab2b-5d2b-487f-bad3-48d44010302f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:00.508817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89ab2b-5d2b-487f-bad3-48d44010302f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:00.509220 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89ab2b-5d2b-487f-bad3-48d44010302f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:00.512084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 468/931] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:00.517582 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3035c-aabb-4102-8545-9e6c31b510a3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:00.517862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3035c-aabb-4102-8545-9e6c31b510a3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:00.518146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3035c-aabb-4102-8545-9e6c31b510a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:00.542899 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-dae3035c-aabb-4102-8545-9e6c31b510a3 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:00.544248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dae3035c-aabb-4102-8545-9e6c31b510a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:00.553105 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849048b-d0fa-451c-afa0-f15d28538ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:00.553978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849048b-d0fa-451c-afa0-f15d28538ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:00.554063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849048b-d0fa-451c-afa0-f15d28538ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:00.554320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849048b-d0fa-451c-afa0-f15d28538ef4 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:00.555010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849048b-d0fa-451c-afa0-f15d28538ef4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:00.556570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849048b-d0fa-451c-afa0-f15d28538ef4 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:00.564355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 464/932] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jun 7 08:23:00 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:23:00.640978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 469/933] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:23:00.665709 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849048b-d0fa-451c-afa0-f15d28538ef4 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:00.792370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849048b-d0fa-451c-afa0-f15d28538ef4 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:00.832858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67acc05-2c16-4287-87ba-78856d0e9251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:00.833575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67acc05-2c16-4287-87ba-78856d0e9251 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:00.833861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67acc05-2c16-4287-87ba-78856d0e9251 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:00.834139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67acc05-2c16-4287-87ba-78856d0e9251 None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:00.834871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67acc05-2c16-4287-87ba-78856d0e9251 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:00.836319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67acc05-2c16-4287-87ba-78856d0e9251 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:00.837727 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: WARNING keystone.server.flask.common [None req-b67acc05-2c16-4287-87ba-78856d0e9251 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 07 08:23:00.860151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.sql.core [None req-b67acc05-2c16-4287-87ba-78856d0e9251 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 08:23:00.860151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 08:23:00.860151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [parameters: {'id': '8a02534c1a6b4b6ea5ddb9b53106d9b5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 08:23:00.860151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80229) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 08:23:00.878320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-b67acc05-2c16-4287-87ba-78856d0e9251 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8a02534c1a6b4b6ea5ddb9b53106d9b5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8a02534c1a6b4b6ea5ddb9b53106d9b5. Jun 07 08:23:00.878320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:00.878320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 08:23:00.878320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 08:23:00.878320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 08:23:00.878320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 08:23:00.878320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 08:23:00.878320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 08:23:00.878320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 08:23:00.878320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application conn.query(q) Jun 07 08:23:00.878320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 08:23:00.878320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 08:23:00.878320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application result.read() Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 07 08:23:00.879237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application next(self.gen) Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application yield resource Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application next(self.gen) Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise self.value Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application session.commit() Jun 07 08:23:00.880225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self.session.flush() Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application compat.raise_( Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 08:23:00.881208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 08:23:00.882200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application conn.query(q) Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application result.read() Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 08:23:00.882998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [parameters: {'id': '8a02534c1a6b4b6ea5ddb9b53106d9b5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:00.883873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:00.884682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:00.891902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:00.891902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 07 08:23:00.891902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 07 08:23:00.891902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:00.891902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:00.891902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 07 08:23:00.891902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 07 08:23:00.891902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8a02534c1a6b4b6ea5ddb9b53106d9b5. Jun 07 08:23:00.891902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:00.891902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 465/934] 10.222.0.39 () {60 vars in 1140 bytes} [Tue Jun 7 08:23:00 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:00.892656 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3035c-aabb-4102-8545-9e6c31b510a3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:00.957351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:01.008405 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1ccc8-75cb-4c10-904b-1f0d8bde1166 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:01.008826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1ccc8-75cb-4c10-904b-1f0d8bde1166 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:01.008932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1ccc8-75cb-4c10-904b-1f0d8bde1166 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:01.018508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 466/935] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:01.021152 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6553a-6b7d-4d74-822e-a68c1d1b6e75 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:01.021769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6553a-6b7d-4d74-822e-a68c1d1b6e75 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:01.022269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6553a-6b7d-4d74-822e-a68c1d1b6e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:01.034471 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3035c-aabb-4102-8545-9e6c31b510a3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:01.047116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-b5d6553a-6b7d-4d74-822e-a68c1d1b6e75 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:01.049420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:01.051803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b5d6553a-6b7d-4d74-822e-a68c1d1b6e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:01.077370 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1c3eb-90e3-4da7-ae6f-1a3a8149383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:01.078867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1c3eb-90e3-4da7-ae6f-1a3a8149383c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:01.079371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1c3eb-90e3-4da7-ae6f-1a3a8149383c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:01.080797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1c3eb-90e3-4da7-ae6f-1a3a8149383c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:01.090384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1c3eb-90e3-4da7-ae6f-1a3a8149383c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:01.092462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1c3eb-90e3-4da7-ae6f-1a3a8149383c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-24a1c3eb-90e3-4da7-ae6f-1a3a8149383c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:23:01.108835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:01.109842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:01.110802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:23:01.110802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:23:01.110802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:23:01.110802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:23:01.110802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:23:01.110802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:23:01.110802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:01.110802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:01.117913 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 470/936] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:01.119207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1c3eb-90e3-4da7-ae6f-1a3a8149383c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:01.169349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 467/937] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:01.193020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6553a-6b7d-4d74-822e-a68c1d1b6e75 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:01.275166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:01.281962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1c3eb-90e3-4da7-ae6f-1a3a8149383c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:01.325868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8beca-9538-4484-bec0-10c62a7504fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:01.326943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8beca-9538-4484-bec0-10c62a7504fb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:01.327449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8beca-9538-4484-bec0-10c62a7504fb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:01.327941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8beca-9538-4484-bec0-10c62a7504fb None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:01.329921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8beca-9538-4484-bec0-10c62a7504fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:01.330512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8beca-9538-4484-bec0-10c62a7504fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:01.332165 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8beca-9538-4484-bec0-10c62a7504fb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:01.358051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 471/938] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jun 7 08:23:01 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:01.371900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6553a-6b7d-4d74-822e-a68c1d1b6e75 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:01.419138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ffb706-aa83-4d13-ae00-1e8d045a0674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:01.420435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffb706-aa83-4d13-ae00-1e8d045a0674 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:01.420941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffb706-aa83-4d13-ae00-1e8d045a0674 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:01.421491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffb706-aa83-4d13-ae00-1e8d045a0674 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:01.425114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:01.436836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffb706-aa83-4d13-ae00-1e8d045a0674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-e1ffb706-aa83-4d13-ae00-1e8d045a0674 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:01.445943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:01.447022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:01.447944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:01.447944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:01.447944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:01.447944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:01.447944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:01.447944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:01.447944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:01.447944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:01.447944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:01.447944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:01.447944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:23:01.447944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:01.452298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 468/939] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Jun 7 08:23:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:01.460622 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8beca-9538-4484-bec0-10c62a7504fb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:01.576797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65f51f-9219-41cc-93ba-78d0e1d6a19b None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:01.577695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65f51f-9219-41cc-93ba-78d0e1d6a19b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:01.578204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65f51f-9219-41cc-93ba-78d0e1d6a19b None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:01.580449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 469/940] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:01.585225 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8beca-9538-4484-bec0-10c62a7504fb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:01.587433 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ea73f-2a47-4e83-a793-72ed1f1f8b7b None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:01.588004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ea73f-2a47-4e83-a793-72ed1f1f8b7b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:01.589048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ea73f-2a47-4e83-a793-72ed1f1f8b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:01.608156 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-159ea73f-2a47-4e83-a793-72ed1f1f8b7b None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:01.609964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-159ea73f-2a47-4e83-a793-72ed1f1f8b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:01.626925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199d264-65b5-4895-94a6-ee42a1547496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:01.627182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199d264-65b5-4895-94a6-ee42a1547496 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:01.627403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199d264-65b5-4895-94a6-ee42a1547496 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:01.627596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199d264-65b5-4895-94a6-ee42a1547496 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:01.628777 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199d264-65b5-4895-94a6-ee42a1547496 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:01.628777 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199d264-65b5-4895-94a6-ee42a1547496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:01.630282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199d264-65b5-4895-94a6-ee42a1547496 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:01.641433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 472/941] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jun 7 08:23:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:01.647554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199d264-65b5-4895-94a6-ee42a1547496 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:01.677830 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:01.710348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 470/942] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:01.736975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ea73f-2a47-4e83-a793-72ed1f1f8b7b None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:01.803232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199d264-65b5-4895-94a6-ee42a1547496 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:01.859708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805292d-6b33-4720-885c-e7e861fb1853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:01.861140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805292d-6b33-4720-885c-e7e861fb1853 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:01.861140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805292d-6b33-4720-885c-e7e861fb1853 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:01.861140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805292d-6b33-4720-885c-e7e861fb1853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:01.875723 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805292d-6b33-4720-885c-e7e861fb1853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:01.877293 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805292d-6b33-4720-885c-e7e861fb1853 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:01.909278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ea73f-2a47-4e83-a793-72ed1f1f8b7b None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-6805292d-6b33-4720-885c-e7e861fb1853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:01.910124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:01.911015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:01.911905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:01.911905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:01.911905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:01.911905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:01.912140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 473/943] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:01.917954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805292d-6b33-4720-885c-e7e861fb1853 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:01.942713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:01.958062 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3f8f8-079f-4927-af2d-3af47ffba606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:01.958882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3f8f8-079f-4927-af2d-3af47ffba606 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:01.959170 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3f8f8-079f-4927-af2d-3af47ffba606 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:01.959722 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3f8f8-079f-4927-af2d-3af47ffba606 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:01.961279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3f8f8-079f-4927-af2d-3af47ffba606 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:01.963221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3f8f8-079f-4927-af2d-3af47ffba606 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-d9b3f8f8-079f-4927-af2d-3af47ffba606 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:01.978867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:01.980918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:01.982623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 471/944] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:01.990868 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3f8f8-079f-4927-af2d-3af47ffba606 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:02.023749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:02.065233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805292d-6b33-4720-885c-e7e861fb1853 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:02.109352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fb7fa-f805-476d-90e7-235863cdc5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:02.110493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fb7fa-f805-476d-90e7-235863cdc5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:02.110989 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fb7fa-f805-476d-90e7-235863cdc5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:02.111502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fb7fa-f805-476d-90e7-235863cdc5c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:02.112587 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fb7fa-f805-476d-90e7-235863cdc5c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:02.113225 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fb7fa-f805-476d-90e7-235863cdc5c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:02.114857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fb7fa-f805-476d-90e7-235863cdc5c5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:02.134788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 474/945] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:02.136233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fb7fa-f805-476d-90e7-235863cdc5c5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:02.146178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3f8f8-079f-4927-af2d-3af47ffba606 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:02.220920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81948b-6899-4bea-8dff-c1fe4c8e17b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:02.222732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81948b-6899-4bea-8dff-c1fe4c8e17b2 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:02.224079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81948b-6899-4bea-8dff-c1fe4c8e17b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:02.224797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81948b-6899-4bea-8dff-c1fe4c8e17b2 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:02.226274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81948b-6899-4bea-8dff-c1fe4c8e17b2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:02.227044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81948b-6899-4bea-8dff-c1fe4c8e17b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:02.230997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81948b-6899-4bea-8dff-c1fe4c8e17b2 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:02.247129 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 472/946] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jun 7 08:23:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:02.249978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81948b-6899-4bea-8dff-c1fe4c8e17b2 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:02.308160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fb7fa-f805-476d-90e7-235863cdc5c5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:02.337462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:02.359617 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:02.367190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5839840-5b7d-4a01-a634-402a642d1a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:02.368945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5839840-5b7d-4a01-a634-402a642d1a40 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:02.370179 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5839840-5b7d-4a01-a634-402a642d1a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:02.370755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5839840-5b7d-4a01-a634-402a642d1a40 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:02.381366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5839840-5b7d-4a01-a634-402a642d1a40 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:02.383806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5839840-5b7d-4a01-a634-402a642d1a40 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:02.398200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81948b-6899-4bea-8dff-c1fe4c8e17b2 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-d5839840-5b7d-4a01-a634-402a642d1a40 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:23:02.405536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:02.406770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:02.407839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:02.407839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:02.407839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:23:02.407839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:23:02.407839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:23:02.407839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:23:02.407839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:23:02.407839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:23:02.407839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:23:02.407839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:02.407839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 475/947] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jun 7 08:23:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:02.412741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5839840-5b7d-4a01-a634-402a642d1a40 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:02.450651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8072973-6d93-48b4-81f7-2c9b597936d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:02.452442 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8072973-6d93-48b4-81f7-2c9b597936d1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:02.452877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8072973-6d93-48b4-81f7-2c9b597936d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:02.453601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8072973-6d93-48b4-81f7-2c9b597936d1 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:02.454483 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8072973-6d93-48b4-81f7-2c9b597936d1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:02.454942 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8072973-6d93-48b4-81f7-2c9b597936d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:02.456486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8072973-6d93-48b4-81f7-2c9b597936d1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:02.469593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 473/948] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jun 7 08:23:02 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:02.539288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5839840-5b7d-4a01-a634-402a642d1a40 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:02.587965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb82cd-27b3-4f64-bd0a-68598648a845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:02.589261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb82cd-27b3-4f64-bd0a-68598648a845 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:02.589638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb82cd-27b3-4f64-bd0a-68598648a845 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:02.589921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb82cd-27b3-4f64-bd0a-68598648a845 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:02.591092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb82cd-27b3-4f64-bd0a-68598648a845 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:02.591405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb82cd-27b3-4f64-bd0a-68598648a845 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:02.593008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb82cd-27b3-4f64-bd0a-68598648a845 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:02.616365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 476/949] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Jun 7 08:23:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:02.625384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8072973-6d93-48b4-81f7-2c9b597936d1 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:02.767713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8072973-6d93-48b4-81f7-2c9b597936d1 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:02.788490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:02.810488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88211d8-225e-4e4b-9ffa-d35d2f666f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:02.811291 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88211d8-225e-4e4b-9ffa-d35d2f666f7f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:02.811574 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88211d8-225e-4e4b-9ffa-d35d2f666f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:02.811856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88211d8-225e-4e4b-9ffa-d35d2f666f7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:02.824781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88211d8-225e-4e4b-9ffa-d35d2f666f7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:02.826345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88211d8-225e-4e4b-9ffa-d35d2f666f7f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-e88211d8-225e-4e4b-9ffa-d35d2f666f7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:02.846676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:02.847645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:02.848528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:02.848528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:02.848528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:02.848528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:02.848528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 474/950] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:02.855487 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb82cd-27b3-4f64-bd0a-68598648a845 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:02.925285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:02.981766 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb82cd-27b3-4f64-bd0a-68598648a845 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:03.020097 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b40d78-99d4-4f73-be55-21e19491ff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:03.020769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b40d78-99d4-4f73-be55-21e19491ff55 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:03.021049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b40d78-99d4-4f73-be55-21e19491ff55 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:03.021345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b40d78-99d4-4f73-be55-21e19491ff55 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:03.022124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b40d78-99d4-4f73-be55-21e19491ff55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:03.022424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b40d78-99d4-4f73-be55-21e19491ff55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:03.023949 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b40d78-99d4-4f73-be55-21e19491ff55 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:03.026231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67e242-c5f5-4577-b454-e9e28203546e None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:03.026711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67e242-c5f5-4577-b454-e9e28203546e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:03.027160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67e242-c5f5-4577-b454-e9e28203546e None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:03.028849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 475/951] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:03.034280 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb20a50-71f1-4f93-9584-8a91ed1605d0 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:03.034570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb20a50-71f1-4f93-9584-8a91ed1605d0 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:03.034847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb20a50-71f1-4f93-9584-8a91ed1605d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:03.037797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 477/952] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:03.044938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b40d78-99d4-4f73-be55-21e19491ff55 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:03.055725 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-3bb20a50-71f1-4f93-9584-8a91ed1605d0 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:03.058123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-3bb20a50-71f1-4f93-9584-8a91ed1605d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:03.157435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 476/953] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:23:03.172679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b40d78-99d4-4f73-be55-21e19491ff55 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:03.183925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb20a50-71f1-4f93-9584-8a91ed1605d0 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:03.187897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:03.219817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134c6d0-f4f4-49a5-a7bd-e5ea8e4ce621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:03.221625 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134c6d0-f4f4-49a5-a7bd-e5ea8e4ce621 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:03.222137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134c6d0-f4f4-49a5-a7bd-e5ea8e4ce621 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:03.223014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134c6d0-f4f4-49a5-a7bd-e5ea8e4ce621 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:03.233919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134c6d0-f4f4-49a5-a7bd-e5ea8e4ce621 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:03.235864 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134c6d0-f4f4-49a5-a7bd-e5ea8e4ce621 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-3134c6d0-f4f4-49a5-a7bd-e5ea8e4ce621 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:23:03.255516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:03.256606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:03.257708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:23:03.257708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:23:03.257708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:23:03.257708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:23:03.257708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:23:03.257708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:23:03.257708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:03.257708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:03.257708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 478/954] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:03.263463 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134c6d0-f4f4-49a5-a7bd-e5ea8e4ce621 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:03.313391 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:03.317349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb20a50-71f1-4f93-9584-8a91ed1605d0 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:03.358691 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9a3ac-ad00-49ad-84e8-8f0f15b9932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:03.360166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9a3ac-ad00-49ad-84e8-8f0f15b9932e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:03.360757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9a3ac-ad00-49ad-84e8-8f0f15b9932e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:03.361352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9a3ac-ad00-49ad-84e8-8f0f15b9932e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:03.362610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9a3ac-ad00-49ad-84e8-8f0f15b9932e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:03.364374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9a3ac-ad00-49ad-84e8-8f0f15b9932e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-65e9a3ac-ad00-49ad-84e8-8f0f15b9932e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:03.379843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:03.381288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:23:03.382524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:03.383841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134c6d0-f4f4-49a5-a7bd-e5ea8e4ce621 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:03.391936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 477/955] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jun 7 08:23:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:03.393298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9a3ac-ad00-49ad-84e8-8f0f15b9932e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:03.424946 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3d0bd-b243-465a-ab85-4c09bd88d941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:03.425731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3d0bd-b243-465a-ab85-4c09bd88d941 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:03.426165 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3d0bd-b243-465a-ab85-4c09bd88d941 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:03.426579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3d0bd-b243-465a-ab85-4c09bd88d941 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:03.427586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3d0bd-b243-465a-ab85-4c09bd88d941 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:03.428009 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3d0bd-b243-465a-ab85-4c09bd88d941 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:03.429576 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3d0bd-b243-465a-ab85-4c09bd88d941 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:03.448512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 479/956] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:23:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:03.449998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3d0bd-b243-465a-ab85-4c09bd88d941 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:03.486566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:03.516730 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9a3ac-ad00-49ad-84e8-8f0f15b9932e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:03.558425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877422ab-8c47-43f8-9014-1b402e796d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:03.559512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877422ab-8c47-43f8-9014-1b402e796d6b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:03.560090 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877422ab-8c47-43f8-9014-1b402e796d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:03.560522 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877422ab-8c47-43f8-9014-1b402e796d6b None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:03.561579 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877422ab-8c47-43f8-9014-1b402e796d6b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:03.562207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877422ab-8c47-43f8-9014-1b402e796d6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:03.564598 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877422ab-8c47-43f8-9014-1b402e796d6b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:03.570783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 478/957] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jun 7 08:23:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:03.571142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3d0bd-b243-465a-ab85-4c09bd88d941 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:03.585589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877422ab-8c47-43f8-9014-1b402e796d6b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:03.612178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025dbf8-9c85-4d73-964a-b3dc1ee8893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:03.613508 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025dbf8-9c85-4d73-964a-b3dc1ee8893c None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:03.613886 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025dbf8-9c85-4d73-964a-b3dc1ee8893c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:03.614285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025dbf8-9c85-4d73-964a-b3dc1ee8893c None admin] PATH_INFO: `/v3/projects/07b861f95bf3498d990785ba72474d82/users/26d6eb06b23b4db88261ae58d650736d/roles/7db380ac539f4b79a45b254e5f57896d` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:03.617069 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025dbf8-9c85-4d73-964a-b3dc1ee8893c None admin] RBAC: Authorizing `identity:create_grant(project_id=07b861f95bf3498d990785ba72474d82, user_id=26d6eb06b23b4db88261ae58d650736d, role_id=7db380ac539f4b79a45b254e5f57896d)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:03.619227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025dbf8-9c85-4d73-964a-b3dc1ee8893c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:03.627760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:03.642061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 480/958] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:23:03 2022] PUT /identity/v3/projects/07b861f95bf3498d990785ba72474d82/users/26d6eb06b23b4db88261ae58d650736d/roles/7db380ac539f4b79a45b254e5f57896d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:23:03.734766 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877422ab-8c47-43f8-9014-1b402e796d6b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:03.778524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd2b56-998f-4dae-b728-1a2fe375efd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:03.779614 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd2b56-998f-4dae-b728-1a2fe375efd1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:03.780102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd2b56-998f-4dae-b728-1a2fe375efd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:03.780600 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd2b56-998f-4dae-b728-1a2fe375efd1 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:03.781565 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd2b56-998f-4dae-b728-1a2fe375efd1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:03.782137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd2b56-998f-4dae-b728-1a2fe375efd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:03.783777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd2b56-998f-4dae-b728-1a2fe375efd1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:03.790395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 479/959] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jun 7 08:23:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:03.797270 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025dbf8-9c85-4d73-964a-b3dc1ee8893c None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:03.929447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:03.941433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025dbf8-9c85-4d73-964a-b3dc1ee8893c None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:04.004303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccff14-ac17-4512-936b-18cca518ab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:04.005426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccff14-ac17-4512-936b-18cca518ab8f None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:04.006022 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccff14-ac17-4512-936b-18cca518ab8f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:04.006552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccff14-ac17-4512-936b-18cca518ab8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:04.007549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccff14-ac17-4512-936b-18cca518ab8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:04.009297 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccff14-ac17-4512-936b-18cca518ab8f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:04.041377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 481/960] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jun 7 08:23:03 2022] POST /identity/v3/endpoints => generated 349 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:23:04.090291 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c30ba-3c2a-4a84-9392-f46679a38947 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:04.091087 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c30ba-3c2a-4a84-9392-f46679a38947 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:04.091087 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c30ba-3c2a-4a84-9392-f46679a38947 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:04.092633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 480/961] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:04.098595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b7333-4ba0-4fd0-988d-099ea34e8d27 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:04.098926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b7333-4ba0-4fd0-988d-099ea34e8d27 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:04.099228 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b7333-4ba0-4fd0-988d-099ea34e8d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:04.124109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3a8b7333-4ba0-4fd0-988d-099ea34e8d27 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:04.126571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3a8b7333-4ba0-4fd0-988d-099ea34e8d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:04.236082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 482/962] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:04.264347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c30ba-3c2a-4a84-9392-f46679a38947 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:04.377166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:04.388787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c30ba-3c2a-4a84-9392-f46679a38947 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:04.424964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32e562-9dd2-4006-9067-d8455cc3d53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:04.425671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32e562-9dd2-4006-9067-d8455cc3d53e None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:04.425950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32e562-9dd2-4006-9067-d8455cc3d53e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:04.426223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32e562-9dd2-4006-9067-d8455cc3d53e None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:04.426917 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32e562-9dd2-4006-9067-d8455cc3d53e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:04.428343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32e562-9dd2-4006-9067-d8455cc3d53e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:04.444198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.sql.core [None req-2b32e562-9dd2-4006-9067-d8455cc3d53e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 08:23:04.444198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 08:23:04.444198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [parameters: {'user_id': '3925d420e8524332a0cb7d77c76cff0d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 08:23:04.444198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80229) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-2b32e562-9dd2-4006-9067-d8455cc3d53e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application conn.query(q) Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application result.read() Jun 07 08:23:04.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 08:23:04.455079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application next(self.gen) Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application yield resource Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application next(self.gen) Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise self.value Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application session.commit() Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 08:23:04.456061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self.session.flush() Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application compat.raise_( Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 08:23:04.456970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 08:23:04.458511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application conn.query(q) Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application result.read() Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 08:23:04.459409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [parameters: {'user_id': '3925d420e8524332a0cb7d77c76cff0d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:04.460252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 07 08:23:04.466446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 07 08:23:04.468758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 07 08:23:04.468758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 07 08:23:04.468758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 08:23:04.468758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 08:23:04.468758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 08:23:04.468758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:04.468758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 481/963] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Jun 7 08:23:04 2022] POST /identity/v3/users => generated 160 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:04.470995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b7333-4ba0-4fd0-988d-099ea34e8d27 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:04.553449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:04.590037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b7333-4ba0-4fd0-988d-099ea34e8d27 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:04.629837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee9704-5dc3-4359-935e-77c7c2cba95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:04.631491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee9704-5dc3-4359-935e-77c7c2cba95a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:04.631986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee9704-5dc3-4359-935e-77c7c2cba95a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:04.632490 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee9704-5dc3-4359-935e-77c7c2cba95a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:04.641595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee9704-5dc3-4359-935e-77c7c2cba95a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:04.643427 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee9704-5dc3-4359-935e-77c7c2cba95a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-2dee9704-5dc3-4359-935e-77c7c2cba95a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:23:04.660725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:04.661758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:04.662710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:04.662710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:04.662710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:23:04.662710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:23:04.662710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:23:04.662710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:23:04.662710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:23:04.662710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:23:04.662710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:23:04.662710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:04.665549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 483/964] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:23:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:04.674779 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32e562-9dd2-4006-9067-d8455cc3d53e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:04.795248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32e562-9dd2-4006-9067-d8455cc3d53e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:04.837278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b713b2-8f68-4a73-9ad9-b6977349476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:04.839031 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b713b2-8f68-4a73-9ad9-b6977349476c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:04.839562 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b713b2-8f68-4a73-9ad9-b6977349476c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:04.840054 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b713b2-8f68-4a73-9ad9-b6977349476c None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:04.841174 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b713b2-8f68-4a73-9ad9-b6977349476c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:04.841730 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b713b2-8f68-4a73-9ad9-b6977349476c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:04.843695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b713b2-8f68-4a73-9ad9-b6977349476c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:04.894092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 482/965] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jun 7 08:23:04 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:05.642622 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae062fd-4e59-4f9e-90fd-98412099a6d9 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:05.643487 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae062fd-4e59-4f9e-90fd-98412099a6d9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:05.643638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae062fd-4e59-4f9e-90fd-98412099a6d9 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:05.646286 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 484/966] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:05.654915 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d86415-cce2-4382-a840-e90db47751c9 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:05.655708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d86415-cce2-4382-a840-e90db47751c9 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:05.655708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d86415-cce2-4382-a840-e90db47751c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:05.675020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-f7d86415-cce2-4382-a840-e90db47751c9 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:05.676561 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f7d86415-cce2-4382-a840-e90db47751c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:05.686761 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:05.773515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 483/967] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:05.815928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae062fd-4e59-4f9e-90fd-98412099a6d9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:05.979602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae062fd-4e59-4f9e-90fd-98412099a6d9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:06.024279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:06.032365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5c56c-253f-40b3-b72a-2d7623e28f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:06.033044 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5c56c-253f-40b3-b72a-2d7623e28f8f None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:06.033360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5c56c-253f-40b3-b72a-2d7623e28f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:06.033637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5c56c-253f-40b3-b72a-2d7623e28f8f None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:06.034384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5c56c-253f-40b3-b72a-2d7623e28f8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:06.035749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5c56c-253f-40b3-b72a-2d7623e28f8f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:06.055736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 485/968] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:23:05 2022] POST /identity/v3/services => generated 242 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:23:06.229411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fd5fa-d37e-4edf-943f-8745b49cedce None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:06.229685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fd5fa-d37e-4edf-943f-8745b49cedce None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:06.230030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fd5fa-d37e-4edf-943f-8745b49cedce None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:06.231774 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 484/969] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:23:06.238904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11614df-c4e7-4ddd-af98-5d4b01bf17dc None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:06.239440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11614df-c4e7-4ddd-af98-5d4b01bf17dc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:06.239944 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11614df-c4e7-4ddd-af98-5d4b01bf17dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:06.259454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b11614df-c4e7-4ddd-af98-5d4b01bf17dc None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:06.261476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b11614df-c4e7-4ddd-af98-5d4b01bf17dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:06.361903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 486/970] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:06.386711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fd5fa-d37e-4edf-943f-8745b49cedce None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:06.562729 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07008e16-da77-4a88-9b34-ccf5780bad68 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:06.562994 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07008e16-da77-4a88-9b34-ccf5780bad68 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:06.563255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07008e16-da77-4a88-9b34-ccf5780bad68 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:06.564539 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 487/971] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:23:06.570086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565114a5-fa5e-42e0-95e0-dfcb6672f7f6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:06.570416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565114a5-fa5e-42e0-95e0-dfcb6672f7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:06.570694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565114a5-fa5e-42e0-95e0-dfcb6672f7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:06.586157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fd5fa-d37e-4edf-943f-8745b49cedce None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:06.588853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-565114a5-fa5e-42e0-95e0-dfcb6672f7f6 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:06.590503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-565114a5-fa5e-42e0-95e0-dfcb6672f7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:06.639538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340281af-e619-42cc-a91d-19b02886f1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:06.640955 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340281af-e619-42cc-a91d-19b02886f1de None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:06.641499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340281af-e619-42cc-a91d-19b02886f1de None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:06.641996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340281af-e619-42cc-a91d-19b02886f1de None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:06.663129 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340281af-e619-42cc-a91d-19b02886f1de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-340281af-e619-42cc-a91d-19b02886f1de None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:06.676253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:06.677296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:06.683675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:06.683675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:06.683675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:06.683675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:06.683675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:06.683675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:06.683675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:06.683675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:06.683675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:06.683675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:06.683675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:23:06.683675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:06.683675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 485/972] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Jun 7 08:23:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:06.684388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340281af-e619-42cc-a91d-19b02886f1de None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:06.689124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 488/973] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:23:06.689892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:06.711491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565114a5-fa5e-42e0-95e0-dfcb6672f7f6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:06.774948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:06.830799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340281af-e619-42cc-a91d-19b02886f1de None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:06.862647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565114a5-fa5e-42e0-95e0-dfcb6672f7f6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:06.891221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a6b04-bb09-4803-852b-393b60711458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:06.892246 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a6b04-bb09-4803-852b-393b60711458 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:06.892645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a6b04-bb09-4803-852b-393b60711458 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:06.892966 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a6b04-bb09-4803-852b-393b60711458 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:06.897255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a6b04-bb09-4803-852b-393b60711458 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:06.897255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a6b04-bb09-4803-852b-393b60711458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:06.897255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a6b04-bb09-4803-852b-393b60711458 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:06.909944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 486/974] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jun 7 08:23:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:06.915535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a6b04-bb09-4803-852b-393b60711458 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:06.929478 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140176a-0461-400d-af02-0f573a758192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:06.931679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140176a-0461-400d-af02-0f573a758192 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:06.932182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140176a-0461-400d-af02-0f573a758192 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:06.932903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140176a-0461-400d-af02-0f573a758192 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:06.935178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140176a-0461-400d-af02-0f573a758192 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:06.938408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140176a-0461-400d-af02-0f573a758192 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-9140176a-0461-400d-af02-0f573a758192 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:06.954085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:06.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:06.956954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 489/975] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jun 7 08:23:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:06.961710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140176a-0461-400d-af02-0f573a758192 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:07.023285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:07.075181 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a6b04-bb09-4803-852b-393b60711458 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:07.109239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140176a-0461-400d-af02-0f573a758192 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:07.112784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13363622-695f-4c7e-88a9-cde31d287f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:07.113619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13363622-695f-4c7e-88a9-cde31d287f23 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:07.113899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13363622-695f-4c7e-88a9-cde31d287f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:07.114174 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13363622-695f-4c7e-88a9-cde31d287f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:07.127137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13363622-695f-4c7e-88a9-cde31d287f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:07.127349 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:07.128702 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13363622-695f-4c7e-88a9-cde31d287f23 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:07.151154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18f4dc-266f-4ef5-a22e-cc51964bd983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-13363622-695f-4c7e-88a9-cde31d287f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:07.151883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:07.152758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:07.153825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:07.153825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:07.153825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:07.153825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:07.154047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18f4dc-266f-4ef5-a22e-cc51964bd983 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:07.154047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18f4dc-266f-4ef5-a22e-cc51964bd983 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:07.154047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18f4dc-266f-4ef5-a22e-cc51964bd983 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:07.154047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18f4dc-266f-4ef5-a22e-cc51964bd983 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:07.154047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18f4dc-266f-4ef5-a22e-cc51964bd983 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:07.159394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 487/976] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:07.159582 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18f4dc-266f-4ef5-a22e-cc51964bd983 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:07.160615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13363622-695f-4c7e-88a9-cde31d287f23 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:07.164091 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 490/977] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jun 7 08:23:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:07.170814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18f4dc-266f-4ef5-a22e-cc51964bd983 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:07.301412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13363622-695f-4c7e-88a9-cde31d287f23 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:07.303894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18f4dc-266f-4ef5-a22e-cc51964bd983 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:07.321025 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:07.347740 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119caf0-2f74-4061-9a4c-1c7c863e98ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:07.348393 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119caf0-2f74-4061-9a4c-1c7c863e98ca None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:07.348678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119caf0-2f74-4061-9a4c-1c7c863e98ca None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:07.348959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119caf0-2f74-4061-9a4c-1c7c863e98ca None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:07.349708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119caf0-2f74-4061-9a4c-1c7c863e98ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:07.350071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119caf0-2f74-4061-9a4c-1c7c863e98ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:07.350371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a755c-d1b1-4812-9716-631e3c41cd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:07.351553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a755c-d1b1-4812-9716-631e3c41cd76 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:07.351678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119caf0-2f74-4061-9a4c-1c7c863e98ca None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:07.352079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a755c-d1b1-4812-9716-631e3c41cd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:07.352585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a755c-d1b1-4812-9716-631e3c41cd76 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:07.354190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a755c-d1b1-4812-9716-631e3c41cd76 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:07.354908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a755c-d1b1-4812-9716-631e3c41cd76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:07.357190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a755c-d1b1-4812-9716-631e3c41cd76 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:07.370868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 491/978] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jun 7 08:23:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:07.371124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 488/979] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:07.374884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a755c-d1b1-4812-9716-631e3c41cd76 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:07.377643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119caf0-2f74-4061-9a4c-1c7c863e98ca None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:07.446875 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:07.516625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119caf0-2f74-4061-9a4c-1c7c863e98ca None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:07.549701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a755c-d1b1-4812-9716-631e3c41cd76 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:07.558919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec48c43-ed31-4ac3-8599-fb4d9388a767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:07.560374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec48c43-ed31-4ac3-8599-fb4d9388a767 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:07.560685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec48c43-ed31-4ac3-8599-fb4d9388a767 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:07.560972 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec48c43-ed31-4ac3-8599-fb4d9388a767 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:07.571200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec48c43-ed31-4ac3-8599-fb4d9388a767 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:07.573324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec48c43-ed31-4ac3-8599-fb4d9388a767 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:07.590492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581fd7d-8942-40b4-ac85-1e91f581a920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:07.591163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581fd7d-8942-40b4-ac85-1e91f581a920 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:07.591445 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581fd7d-8942-40b4-ac85-1e91f581a920 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:07.591718 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581fd7d-8942-40b4-ac85-1e91f581a920 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:07.592539 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581fd7d-8942-40b4-ac85-1e91f581a920 None admin] RBAC: Adding query filter params (interface=admin, service_id=98f2668e73c546d0a442150e4596095a, region_id=RegionOne) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:07.592960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581fd7d-8942-40b4-ac85-1e91f581a920 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-8ec48c43-ed31-4ac3-8599-fb4d9388a767 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:23:07.593627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:07.595221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:07.596666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:07.596666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:07.596666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:23:07.596666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:23:07.596666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:23:07.596666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:23:07.596666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:23:07.596666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:23:07.596666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:23:07.596666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:07.597760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581fd7d-8942-40b4-ac85-1e91f581a920 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:07.597899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 489/980] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jun 7 08:23:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:07.600839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec48c43-ed31-4ac3-8599-fb4d9388a767 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:07.612295 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 492/981] 10.222.0.39 () {56 vars in 1249 bytes} [Tue Jun 7 08:23:07 2022] GET /identity/v3/endpoints?service_id=98f2668e73c546d0a442150e4596095a&interface=admin®ion_id=RegionOne => generated 193 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:07.616252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581fd7d-8942-40b4-ac85-1e91f581a920 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:07.648377 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:07.753479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec48c43-ed31-4ac3-8599-fb4d9388a767 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:07.755517 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581fd7d-8942-40b4-ac85-1e91f581a920 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:07.759680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:07.795359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d2759-6937-4716-865b-8ce520196175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:07.796188 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d2759-6937-4716-865b-8ce520196175 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:07.796473 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d2759-6937-4716-865b-8ce520196175 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:07.796754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d2759-6937-4716-865b-8ce520196175 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:07.797658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d2759-6937-4716-865b-8ce520196175 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:07.797971 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d2759-6937-4716-865b-8ce520196175 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:07.799446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d2759-6937-4716-865b-8ce520196175 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:07.820770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6049cb3-0caa-4721-8ba6-78db38ec7b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:07.821590 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6049cb3-0caa-4721-8ba6-78db38ec7b79 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:07.829903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6049cb3-0caa-4721-8ba6-78db38ec7b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:07.829903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6049cb3-0caa-4721-8ba6-78db38ec7b79 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:07.830324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 490/982] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Jun 7 08:23:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:07.830469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6049cb3-0caa-4721-8ba6-78db38ec7b79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:07.830469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6049cb3-0caa-4721-8ba6-78db38ec7b79 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:07.833197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d2759-6937-4716-865b-8ce520196175 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:07.842447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 493/983] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jun 7 08:23:07 2022] GET /identity/v3/services => generated 1727 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:23:08.026983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:08.041551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d2759-6937-4716-865b-8ce520196175 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:08.100256 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676692d-dd29-46b0-99bd-44928d90d8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:08.100925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676692d-dd29-46b0-99bd-44928d90d8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:08.101294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676692d-dd29-46b0-99bd-44928d90d8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:08.101576 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676692d-dd29-46b0-99bd-44928d90d8e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:08.179485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676692d-dd29-46b0-99bd-44928d90d8e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:08.180888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676692d-dd29-46b0-99bd-44928d90d8e4 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:08.322758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:08.336191 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 491/984] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:07 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 08:23:08.345588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6049cb3-0caa-4721-8ba6-78db38ec7b79 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:08.488709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6049cb3-0caa-4721-8ba6-78db38ec7b79 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:08.528806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24851147-960f-4bb7-877d-dbaa7aaadb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:08.529938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24851147-960f-4bb7-877d-dbaa7aaadb55 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:08.530423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24851147-960f-4bb7-877d-dbaa7aaadb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:08.530873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24851147-960f-4bb7-877d-dbaa7aaadb55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:08.543569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24851147-960f-4bb7-877d-dbaa7aaadb55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:08.545352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24851147-960f-4bb7-877d-dbaa7aaadb55 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-24851147-960f-4bb7-877d-dbaa7aaadb55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:08.567302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:08.568260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:08.569199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:08.569199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:08.569199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:08.569199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:08.576567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 494/985] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:08.577971 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676692d-dd29-46b0-99bd-44928d90d8e4 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:08.667411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58dd35-73a6-4456-a5a1-6130747100fd None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:08.667743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58dd35-73a6-4456-a5a1-6130747100fd None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:08.668084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58dd35-73a6-4456-a5a1-6130747100fd None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:08.669695 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 495/986] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:23:08.674752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b5e79-2d07-4cb8-96b1-7021ca141d6c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:08.675038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b5e79-2d07-4cb8-96b1-7021ca141d6c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:08.675320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b5e79-2d07-4cb8-96b1-7021ca141d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:08.693754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c86b5e79-2d07-4cb8-96b1-7021ca141d6c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:08.694964 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c86b5e79-2d07-4cb8-96b1-7021ca141d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:08.715719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:08.731692 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676692d-dd29-46b0-99bd-44928d90d8e4 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:08.772932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68dc13-b393-4d08-b279-9a4983ed459f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:08.773817 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68dc13-b393-4d08-b279-9a4983ed459f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:08.774092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68dc13-b393-4d08-b279-9a4983ed459f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:08.774372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68dc13-b393-4d08-b279-9a4983ed459f None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:08.775999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68dc13-b393-4d08-b279-9a4983ed459f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:08.776312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68dc13-b393-4d08-b279-9a4983ed459f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:08.777722 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68dc13-b393-4d08-b279-9a4983ed459f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:08.791450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 492/987] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:08.796998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68dc13-b393-4d08-b279-9a4983ed459f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:08.813383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 496/988] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:23:08.827322 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b5e79-2d07-4cb8-96b1-7021ca141d6c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:08.875414 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:08.931390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68dc13-b393-4d08-b279-9a4983ed459f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:08.974394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443116c-b1ca-4eba-9110-0c8a8051ae26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:08.975371 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443116c-b1ca-4eba-9110-0c8a8051ae26 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:08.975699 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443116c-b1ca-4eba-9110-0c8a8051ae26 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:08.975999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443116c-b1ca-4eba-9110-0c8a8051ae26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:08.985716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b5e79-2d07-4cb8-96b1-7021ca141d6c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:08.986015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443116c-b1ca-4eba-9110-0c8a8051ae26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:08.988832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443116c-b1ca-4eba-9110-0c8a8051ae26 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-2443116c-b1ca-4eba-9110-0c8a8051ae26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:23:09.019306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:09.020274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:09.021206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:23:09.021206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:23:09.021206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:23:09.021206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:23:09.021206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:23:09.021206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:23:09.021206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:09.021206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:09.022102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 493/989] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:09.027750 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443116c-b1ca-4eba-9110-0c8a8051ae26 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:09.032761 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefa9cb-be21-4a49-be12-6ad71f4548a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:09.033603 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefa9cb-be21-4a49-be12-6ad71f4548a9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:09.033883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefa9cb-be21-4a49-be12-6ad71f4548a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:09.034164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefa9cb-be21-4a49-be12-6ad71f4548a9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:09.034825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefa9cb-be21-4a49-be12-6ad71f4548a9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:09.036205 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefa9cb-be21-4a49-be12-6ad71f4548a9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-eaefa9cb-be21-4a49-be12-6ad71f4548a9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:09.050128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:09.051268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:09.052167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 497/990] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:09.058486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefa9cb-be21-4a49-be12-6ad71f4548a9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:09.078676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:09.180939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443116c-b1ca-4eba-9110-0c8a8051ae26 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:09.200719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefa9cb-be21-4a49-be12-6ad71f4548a9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:09.210383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:09.246281 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d7506-1a43-4a60-ac6f-69cec9144f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:09.247237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d7506-1a43-4a60-ac6f-69cec9144f9b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:09.247659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d7506-1a43-4a60-ac6f-69cec9144f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:09.248180 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d7506-1a43-4a60-ac6f-69cec9144f9b None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:09.248645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8043aed-ffc1-4d79-bd40-67fe9bc1b837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:09.249263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d7506-1a43-4a60-ac6f-69cec9144f9b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:09.249378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8043aed-ffc1-4d79-bd40-67fe9bc1b837 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:09.249605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8043aed-ffc1-4d79-bd40-67fe9bc1b837 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:09.249879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8043aed-ffc1-4d79-bd40-67fe9bc1b837 None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:09.250118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d7506-1a43-4a60-ac6f-69cec9144f9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:09.250677 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8043aed-ffc1-4d79-bd40-67fe9bc1b837 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:09.250984 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8043aed-ffc1-4d79-bd40-67fe9bc1b837 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:09.251927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d7506-1a43-4a60-ac6f-69cec9144f9b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:09.252418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8043aed-ffc1-4d79-bd40-67fe9bc1b837 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:09.260911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 498/991] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jun 7 08:23:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:09.266707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d7506-1a43-4a60-ac6f-69cec9144f9b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:09.268847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 494/992] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:23:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:09.276659 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8043aed-ffc1-4d79-bd40-67fe9bc1b837 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:09.387583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:09.408814 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8043aed-ffc1-4d79-bd40-67fe9bc1b837 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:09.413601 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d7506-1a43-4a60-ac6f-69cec9144f9b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:09.450864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386e891-e219-4be8-9d8b-53aca655b468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:09.451887 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386e891-e219-4be8-9d8b-53aca655b468 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:09.452362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386e891-e219-4be8-9d8b-53aca655b468 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:09.452836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386e891-e219-4be8-9d8b-53aca655b468 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:09.454275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386e891-e219-4be8-9d8b-53aca655b468 None admin] RBAC: Adding query filter params (role.id=7db380ac539f4b79a45b254e5f57896d, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=26d6eb06b23b4db88261ae58d650736d) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:09.454855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386e891-e219-4be8-9d8b-53aca655b468 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:09.456620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386e891-e219-4be8-9d8b-53aca655b468 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:09.458633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152701d-e4cd-4950-8c7c-f284481949ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:09.459297 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152701d-e4cd-4950-8c7c-f284481949ea None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:09.459581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152701d-e4cd-4950-8c7c-f284481949ea None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:09.459857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152701d-e4cd-4950-8c7c-f284481949ea None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:09.460564 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152701d-e4cd-4950-8c7c-f284481949ea None admin] RBAC: Adding query filter params (type=compute) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:09.460883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152701d-e4cd-4950-8c7c-f284481949ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:09.462270 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152701d-e4cd-4950-8c7c-f284481949ea None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:09.465037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 495/993] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:23:09 2022] GET /identity/v3/role_assignments?user.id=26d6eb06b23b4db88261ae58d650736d&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=7db380ac539f4b79a45b254e5f57896d => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:09.468701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 499/994] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jun 7 08:23:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:09.480070 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386e891-e219-4be8-9d8b-53aca655b468 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:09.555616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:09.669949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386e891-e219-4be8-9d8b-53aca655b468 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:09.715543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798056f-8a4d-45bb-bab7-b3a4f412fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:09.716215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798056f-8a4d-45bb-bab7-b3a4f412fee8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:09.716507 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798056f-8a4d-45bb-bab7-b3a4f412fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:09.716801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798056f-8a4d-45bb-bab7-b3a4f412fee8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:09.717624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798056f-8a4d-45bb-bab7-b3a4f412fee8 None admin] RBAC: Adding query filter params (interface=public, service_id=408dd623ce6745b88e396258fb74ae6f, region_id=RegionOne) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:09.717933 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798056f-8a4d-45bb-bab7-b3a4f412fee8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:09.719322 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798056f-8a4d-45bb-bab7-b3a4f412fee8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:09.725625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 496/995] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jun 7 08:23:09 2022] GET /identity/v3/endpoints?service_id=408dd623ce6745b88e396258fb74ae6f&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:09.734046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152701d-e4cd-4950-8c7c-f284481949ea None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:09.852062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152701d-e4cd-4950-8c7c-f284481949ea None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:09.908036 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f4261-3062-41ef-9e25-9c7090691630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:09.908727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f4261-3062-41ef-9e25-9c7090691630 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:09.909014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f4261-3062-41ef-9e25-9c7090691630 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:09.909298 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f4261-3062-41ef-9e25-9c7090691630 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:09.909999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f4261-3062-41ef-9e25-9c7090691630 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:09.911403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f4261-3062-41ef-9e25-9c7090691630 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:09.921185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 500/996] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jun 7 08:23:09 2022] GET /identity/v3/services => generated 1727 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:23:10.249888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b94759-eceb-4c10-8d06-2bc5343b1cf5 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:10.250389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b94759-eceb-4c10-8d06-2bc5343b1cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:10.250389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b94759-eceb-4c10-8d06-2bc5343b1cf5 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:10.251751 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 497/997] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:10.257679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a381fa5-ea7e-4090-b0dd-15216df39b40 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:10.257980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a381fa5-ea7e-4090-b0dd-15216df39b40 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:10.258247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a381fa5-ea7e-4090-b0dd-15216df39b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:10.272099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:10.278949 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3a381fa5-ea7e-4090-b0dd-15216df39b40 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:10.280195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3a381fa5-ea7e-4090-b0dd-15216df39b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:10.376699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 501/998] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:10.401642 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b94759-eceb-4c10-8d06-2bc5343b1cf5 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:10.521728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b94759-eceb-4c10-8d06-2bc5343b1cf5 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:10.523261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:10.574587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e83dc-456c-49b7-bc9c-d3ad7ade1c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:10.575397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e83dc-456c-49b7-bc9c-d3ad7ade1c99 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:10.575684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e83dc-456c-49b7-bc9c-d3ad7ade1c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:10.575957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e83dc-456c-49b7-bc9c-d3ad7ade1c99 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:10.576607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e83dc-456c-49b7-bc9c-d3ad7ade1c99 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:10.578013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e83dc-456c-49b7-bc9c-d3ad7ade1c99 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-ce8e83dc-456c-49b7-bc9c-d3ad7ade1c99 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:10.591958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:10.593211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:23:10.594077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:10.597416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 498/999] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jun 7 08:23:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:10.599492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a381fa5-ea7e-4090-b0dd-15216df39b40 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:10.757430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a381fa5-ea7e-4090-b0dd-15216df39b40 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:10.801261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d997aac-76b5-4e29-9b03-4586c816bf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:10.801873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d997aac-76b5-4e29-9b03-4586c816bf60 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:10.802137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d997aac-76b5-4e29-9b03-4586c816bf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:10.802492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d997aac-76b5-4e29-9b03-4586c816bf60 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:10.803288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d997aac-76b5-4e29-9b03-4586c816bf60 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:10.803601 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d997aac-76b5-4e29-9b03-4586c816bf60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:10.805057 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d997aac-76b5-4e29-9b03-4586c816bf60 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:10.808011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:10.818278 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 502/1000] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jun 7 08:23:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:10.819924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e83dc-456c-49b7-bc9c-d3ad7ade1c99 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:10.980347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e83dc-456c-49b7-bc9c-d3ad7ade1c99 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:11.042270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f257a-4eb5-4ee0-b3bc-f76d98ef5b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:11.043450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f257a-4eb5-4ee0-b3bc-f76d98ef5b6b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:11.043982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f257a-4eb5-4ee0-b3bc-f76d98ef5b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:11.044496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f257a-4eb5-4ee0-b3bc-f76d98ef5b6b None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:11.045943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f257a-4eb5-4ee0-b3bc-f76d98ef5b6b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:11.046521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f257a-4eb5-4ee0-b3bc-f76d98ef5b6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:11.049222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f257a-4eb5-4ee0-b3bc-f76d98ef5b6b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:11.065259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 499/1001] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jun 7 08:23:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:11.066360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d997aac-76b5-4e29-9b03-4586c816bf60 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:11.212784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d997aac-76b5-4e29-9b03-4586c816bf60 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:11.250614 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb17f30-ca1f-4d91-a7fa-5445a5e310c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:11.251275 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb17f30-ca1f-4d91-a7fa-5445a5e310c6 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:11.251560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb17f30-ca1f-4d91-a7fa-5445a5e310c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:11.251840 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb17f30-ca1f-4d91-a7fa-5445a5e310c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:11.252600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb17f30-ca1f-4d91-a7fa-5445a5e310c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:11.254014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb17f30-ca1f-4d91-a7fa-5445a5e310c6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:11.273188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 503/1002] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jun 7 08:23:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:23:11.906741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452eb30-0201-4f16-8954-555ba5352293 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:11.907313 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452eb30-0201-4f16-8954-555ba5352293 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:11.907790 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452eb30-0201-4f16-8954-555ba5352293 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:11.910772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 500/1003] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:11.920226 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8990f-29b3-49e8-ab56-c2e720000214 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:11.920560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8990f-29b3-49e8-ab56-c2e720000214 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:11.920883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8990f-29b3-49e8-ab56-c2e720000214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:11.943661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-aee8990f-29b3-49e8-ab56-c2e720000214 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:11.944957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-aee8990f-29b3-49e8-ab56-c2e720000214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:12.018143 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:12.054446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 504/1004] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:11 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:12.091060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452eb30-0201-4f16-8954-555ba5352293 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:12.139466 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:12.234402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452eb30-0201-4f16-8954-555ba5352293 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:12.272292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879be88b-aa5f-4a3a-ba33-133c2803374f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:12.273112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879be88b-aa5f-4a3a-ba33-133c2803374f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:12.273415 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879be88b-aa5f-4a3a-ba33-133c2803374f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:12.273696 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879be88b-aa5f-4a3a-ba33-133c2803374f None admin] PATH_INFO: `/v3/services/image` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:12.274366 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879be88b-aa5f-4a3a-ba33-133c2803374f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:12.276083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879be88b-aa5f-4a3a-ba33-133c2803374f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-879be88b-aa5f-4a3a-ba33-133c2803374f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:12.288918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:12.290037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:12.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 501/1005] 10.222.0.39 () {56 vars in 1105 bytes} [Tue Jun 7 08:23:12 2022] GET /identity/v3/services/image => generated 86 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:12.297446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8990f-29b3-49e8-ab56-c2e720000214 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:12.310062 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd896f71-1f65-489c-99b4-dc5a9bb502b3 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:12.310911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd896f71-1f65-489c-99b4-dc5a9bb502b3 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:12.311419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd896f71-1f65-489c-99b4-dc5a9bb502b3 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:12.314947 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 502/1006] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:12.319352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b95165-269c-4bbd-a979-5e2d1a2ed6ef None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:12.319646 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b95165-269c-4bbd-a979-5e2d1a2ed6ef None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:12.319932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b95165-269c-4bbd-a979-5e2d1a2ed6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:12.340166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-79b95165-269c-4bbd-a979-5e2d1a2ed6ef None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:12.341502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-79b95165-269c-4bbd-a979-5e2d1a2ed6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:12.423157 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8990f-29b3-49e8-ab56-c2e720000214 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:12.457152 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 503/1007] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:12 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:23:12.475597 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e712bd-1edb-4694-b598-d12089969006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:12.476801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e712bd-1edb-4694-b598-d12089969006 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:12.477369 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e712bd-1edb-4694-b598-d12089969006 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:12.477897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e712bd-1edb-4694-b598-d12089969006 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:12.479314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e712bd-1edb-4694-b598-d12089969006 None admin] RBAC: Adding query filter params (name=image) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:12.479814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e712bd-1edb-4694-b598-d12089969006 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:12.482302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e712bd-1edb-4694-b598-d12089969006 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:12.482515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b95165-269c-4bbd-a979-5e2d1a2ed6ef None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:12.488151 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 505/1008] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:12.493485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e712bd-1edb-4694-b598-d12089969006 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:12.515488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:12.586865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:12.607362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b95165-269c-4bbd-a979-5e2d1a2ed6ef None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:12.649112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba0902-7aed-4839-a152-e05023ead84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:12.650116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba0902-7aed-4839-a152-e05023ead84e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:12.650592 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba0902-7aed-4839-a152-e05023ead84e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:12.651047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba0902-7aed-4839-a152-e05023ead84e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:12.652085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba0902-7aed-4839-a152-e05023ead84e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:12.653831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba0902-7aed-4839-a152-e05023ead84e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:12.659143 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e712bd-1edb-4694-b598-d12089969006 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-d0ba0902-7aed-4839-a152-e05023ead84e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:12.674242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:12.675331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:12.682205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 504/1009] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:12.683130 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba0902-7aed-4839-a152-e05023ead84e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:12.719406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7747d7-1e4b-4df1-a7a8-6e8bcab6ca20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:12.720082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7747d7-1e4b-4df1-a7a8-6e8bcab6ca20 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:12.720363 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7747d7-1e4b-4df1-a7a8-6e8bcab6ca20 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:12.720637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7747d7-1e4b-4df1-a7a8-6e8bcab6ca20 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:12.721499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7747d7-1e4b-4df1-a7a8-6e8bcab6ca20 None admin] RBAC: Adding query filter params (type=image) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:12.721686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7747d7-1e4b-4df1-a7a8-6e8bcab6ca20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:12.723041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7747d7-1e4b-4df1-a7a8-6e8bcab6ca20 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:12.729045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 506/1010] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:12.846378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:12.863334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba0902-7aed-4839-a152-e05023ead84e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:12.935224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1900b2a-7335-4f7a-aed5-006ffa7579a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:12.936245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1900b2a-7335-4f7a-aed5-006ffa7579a1 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:12.936558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1900b2a-7335-4f7a-aed5-006ffa7579a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:12.937018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1900b2a-7335-4f7a-aed5-006ffa7579a1 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:12.938362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1900b2a-7335-4f7a-aed5-006ffa7579a1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:12.938931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1900b2a-7335-4f7a-aed5-006ffa7579a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:12.941624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1900b2a-7335-4f7a-aed5-006ffa7579a1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:12.948994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 505/1011] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jun 7 08:23:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:12.959020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7747d7-1e4b-4df1-a7a8-6e8bcab6ca20 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:13.091432 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7747d7-1e4b-4df1-a7a8-6e8bcab6ca20 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:13.119013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:13.144344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf151e-6318-4f11-8c30-4e8e0991304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:13.145014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf151e-6318-4f11-8c30-4e8e0991304f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:13.145399 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf151e-6318-4f11-8c30-4e8e0991304f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:13.145675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf151e-6318-4f11-8c30-4e8e0991304f None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:13.146384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf151e-6318-4f11-8c30-4e8e0991304f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:13.146690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf151e-6318-4f11-8c30-4e8e0991304f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:13.148045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf151e-6318-4f11-8c30-4e8e0991304f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:13.154986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 507/1012] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jun 7 08:23:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:13.161810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1900b2a-7335-4f7a-aed5-006ffa7579a1 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:13.305980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1900b2a-7335-4f7a-aed5-006ffa7579a1 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:13.349217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4310ad-6e43-4142-af7f-c27623460714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:13.350203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4310ad-6e43-4142-af7f-c27623460714 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:13.350538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4310ad-6e43-4142-af7f-c27623460714 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:13.350999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4310ad-6e43-4142-af7f-c27623460714 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:13.351913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4310ad-6e43-4142-af7f-c27623460714 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:13.354261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4310ad-6e43-4142-af7f-c27623460714 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:13.389204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 506/1013] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jun 7 08:23:13 2022] POST /identity/v3/endpoints => generated 334 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:23:13.775009 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c2085-c78d-4dd1-8a35-2810fa766d41 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:13.775721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c2085-c78d-4dd1-8a35-2810fa766d41 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:13.775721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c2085-c78d-4dd1-8a35-2810fa766d41 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:13.777202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 508/1014] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:13.783290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cb4a5-5248-4184-b1ba-c2c5b2b80b2c None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:13.783580 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cb4a5-5248-4184-b1ba-c2c5b2b80b2c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:13.783862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cb4a5-5248-4184-b1ba-c2c5b2b80b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:13.801981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-9d3cb4a5-5248-4184-b1ba-c2c5b2b80b2c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:13.803191 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9d3cb4a5-5248-4184-b1ba-c2c5b2b80b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:13.805676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:13.913189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 507/1015] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:13 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:13.937642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c2085-c78d-4dd1-8a35-2810fa766d41 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:14.105124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c2085-c78d-4dd1-8a35-2810fa766d41 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:14.170764 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d8a83-ab44-497f-afd9-dd7a41a23777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:14.172181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d8a83-ab44-497f-afd9-dd7a41a23777 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:14.172697 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d8a83-ab44-497f-afd9-dd7a41a23777 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:14.173401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d8a83-ab44-497f-afd9-dd7a41a23777 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:14.175550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d8a83-ab44-497f-afd9-dd7a41a23777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:14.178399 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d8a83-ab44-497f-afd9-dd7a41a23777 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:14.185313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 509/1016] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:14.188843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cb4a5-5248-4184-b1ba-c2c5b2b80b2c None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:14.352115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cb4a5-5248-4184-b1ba-c2c5b2b80b2c None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:14.396832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bf6fc-1b06-41a9-82a3-d748de6469ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:14.397628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bf6fc-1b06-41a9-82a3-d748de6469ee None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:14.397915 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bf6fc-1b06-41a9-82a3-d748de6469ee None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:14.398195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bf6fc-1b06-41a9-82a3-d748de6469ee None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:14.399015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3bf6fc-1b06-41a9-82a3-d748de6469ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:14.400707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3bf6fc-1b06-41a9-82a3-d748de6469ee None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:14.404328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:14.441142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 508/1017] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:23:14 2022] POST /identity/v3/projects => generated 311 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:23:15.109224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71651c30-1963-443f-99c9-762a3ed6976d None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:15.110036 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71651c30-1963-443f-99c9-762a3ed6976d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:15.110036 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71651c30-1963-443f-99c9-762a3ed6976d None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:15.111378 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 510/1018] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:23:15.116737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cad46c-6cd1-4edc-ba3e-540a12ed7e13 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:15.117036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cad46c-6cd1-4edc-ba3e-540a12ed7e13 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:15.117359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cad46c-6cd1-4edc-ba3e-540a12ed7e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:15.135498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-63cad46c-6cd1-4edc-ba3e-540a12ed7e13 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:15.136869 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-63cad46c-6cd1-4edc-ba3e-540a12ed7e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:15.222771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 509/1019] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:15 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:15.252987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71651c30-1963-443f-99c9-762a3ed6976d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:15.311266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:15.394604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71651c30-1963-443f-99c9-762a3ed6976d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:15.459827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3076bfa-b107-4c60-a8cf-a7b5050f4af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:15.461044 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3076bfa-b107-4c60-a8cf-a7b5050f4af8 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:15.461611 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3076bfa-b107-4c60-a8cf-a7b5050f4af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:15.462132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3076bfa-b107-4c60-a8cf-a7b5050f4af8 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:15.464129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3076bfa-b107-4c60-a8cf-a7b5050f4af8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:15.466871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3076bfa-b107-4c60-a8cf-a7b5050f4af8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:15.486392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 511/1020] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:23:15 2022] POST /identity/v3/services => generated 242 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:23:15.977352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72166823-bd3d-4004-b02c-e9624477b898 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:15.977616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72166823-bd3d-4004-b02c-e9624477b898 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:15.977898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72166823-bd3d-4004-b02c-e9624477b898 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:15.979584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 510/1021] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:23:15.984925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d55c20-5cb8-434b-b50a-66d927ed8f04 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:15.985262 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d55c20-5cb8-434b-b50a-66d927ed8f04 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:15.985538 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d55c20-5cb8-434b-b50a-66d927ed8f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:16.002533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-75d55c20-5cb8-434b-b50a-66d927ed8f04 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:16.003740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-75d55c20-5cb8-434b-b50a-66d927ed8f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:16.090930 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 512/1022] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:16.115256 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72166823-bd3d-4004-b02c-e9624477b898 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:16.268577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72166823-bd3d-4004-b02c-e9624477b898 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:16.290338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:16.327950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53f43d-0cf3-4584-869f-d2d34bfa714d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:16.329698 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53f43d-0cf3-4584-869f-d2d34bfa714d None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:16.330235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53f43d-0cf3-4584-869f-d2d34bfa714d None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:16.330757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53f43d-0cf3-4584-869f-d2d34bfa714d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:16.351773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53f43d-0cf3-4584-869f-d2d34bfa714d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-bd53f43d-0cf3-4584-869f-d2d34bfa714d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:16.369902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:16.371589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:16.373367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:16.373367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:16.373367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:16.373367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:16.373367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:16.373367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:16.373367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:16.373367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:16.373367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:16.373367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:16.373367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:23:16.373367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:16.375629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 511/1023] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Jun 7 08:23:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:16.382761 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d55c20-5cb8-434b-b50a-66d927ed8f04 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:16.430365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:16.521878 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d55c20-5cb8-434b-b50a-66d927ed8f04 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:16.570023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d476e20-30b5-4810-a2b3-e3a3cd1198fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:16.570815 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d476e20-30b5-4810-a2b3-e3a3cd1198fe None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:16.571091 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d476e20-30b5-4810-a2b3-e3a3cd1198fe None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:16.571362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d476e20-30b5-4810-a2b3-e3a3cd1198fe None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:16.572143 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d476e20-30b5-4810-a2b3-e3a3cd1198fe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:16.572496 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d476e20-30b5-4810-a2b3-e3a3cd1198fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:16.573893 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d476e20-30b5-4810-a2b3-e3a3cd1198fe None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:16.587387 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 513/1024] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jun 7 08:23:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:16.589827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53f43d-0cf3-4584-869f-d2d34bfa714d None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:16.756500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53f43d-0cf3-4584-869f-d2d34bfa714d None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:16.811911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58306ce-9bc5-4944-ad88-2ed943d73d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:16.812690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58306ce-9bc5-4944-ad88-2ed943d73d0f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:16.813194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58306ce-9bc5-4944-ad88-2ed943d73d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:16.813555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58306ce-9bc5-4944-ad88-2ed943d73d0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:16.827759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58306ce-9bc5-4944-ad88-2ed943d73d0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:16.829551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58306ce-9bc5-4944-ad88-2ed943d73d0f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-b58306ce-9bc5-4944-ad88-2ed943d73d0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:16.848679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:16.850799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:16.852801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:16.852801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:16.852801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:16.852801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:16.852801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 512/1025] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:16.861137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d476e20-30b5-4810-a2b3-e3a3cd1198fe None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:16.866023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc3d13-80fa-4387-8c8e-469ff40b5ec9 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:16.866307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc3d13-80fa-4387-8c8e-469ff40b5ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:16.866577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc3d13-80fa-4387-8c8e-469ff40b5ec9 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:16.868231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 513/1026] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:23:16.874491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5e5da-dc0e-4d89-8021-c4960658f651 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:16.874787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5e5da-dc0e-4d89-8021-c4960658f651 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:16.875055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5e5da-dc0e-4d89-8021-c4960658f651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:16.896459 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-44a5e5da-dc0e-4d89-8021-c4960658f651 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:16.897766 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-44a5e5da-dc0e-4d89-8021-c4960658f651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:16.934933 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:17.017985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 514/1027] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:23:17.046684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:17.055537 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5e5da-dc0e-4d89-8021-c4960658f651 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:17.061867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d476e20-30b5-4810-a2b3-e3a3cd1198fe None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:17.105613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60d7a2-b92c-48cd-98c9-b0bfed14d90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:17.106210 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60d7a2-b92c-48cd-98c9-b0bfed14d90d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:17.106476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60d7a2-b92c-48cd-98c9-b0bfed14d90d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:17.106759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60d7a2-b92c-48cd-98c9-b0bfed14d90d None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:17.107502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a60d7a2-b92c-48cd-98c9-b0bfed14d90d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:17.107810 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a60d7a2-b92c-48cd-98c9-b0bfed14d90d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:17.109251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a60d7a2-b92c-48cd-98c9-b0bfed14d90d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:17.125119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 514/1028] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:17.130940 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60d7a2-b92c-48cd-98c9-b0bfed14d90d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:17.233227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5e5da-dc0e-4d89-8021-c4960658f651 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:17.303005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039647d-75da-4942-bb9a-212bf4c06aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:17.304203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039647d-75da-4942-bb9a-212bf4c06aec None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:17.304587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039647d-75da-4942-bb9a-212bf4c06aec None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:17.305079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039647d-75da-4942-bb9a-212bf4c06aec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:17.307861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039647d-75da-4942-bb9a-212bf4c06aec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:17.309514 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039647d-75da-4942-bb9a-212bf4c06aec None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:17.311258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 515/1029] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:17.312625 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60d7a2-b92c-48cd-98c9-b0bfed14d90d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:17.318482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039647d-75da-4942-bb9a-212bf4c06aec None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:17.351131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:17.369920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650913b5-7d83-4d77-86e0-d0a868c85c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:17.370734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650913b5-7d83-4d77-86e0-d0a868c85c2d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:17.371003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650913b5-7d83-4d77-86e0-d0a868c85c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:17.371313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650913b5-7d83-4d77-86e0-d0a868c85c2d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:17.380386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650913b5-7d83-4d77-86e0-d0a868c85c2d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:17.382141 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650913b5-7d83-4d77-86e0-d0a868c85c2d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-650913b5-7d83-4d77-86e0-d0a868c85c2d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:23:17.401362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:17.402826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:17.403965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:17.403965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:17.403965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:23:17.403965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:23:17.403965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:23:17.403965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:23:17.403965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:23:17.403965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:23:17.403965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:23:17.403965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:17.403965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 515/1030] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jun 7 08:23:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:17.412618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650913b5-7d83-4d77-86e0-d0a868c85c2d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:17.434208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:17.453893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039647d-75da-4942-bb9a-212bf4c06aec None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:17.503897 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72aaab-f8c4-4343-a069-1501b5587055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:17.504878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72aaab-f8c4-4343-a069-1501b5587055 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:17.505322 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72aaab-f8c4-4343-a069-1501b5587055 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:17.506730 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72aaab-f8c4-4343-a069-1501b5587055 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:17.507019 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72aaab-f8c4-4343-a069-1501b5587055 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:17.509003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72aaab-f8c4-4343-a069-1501b5587055 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:17.532616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 516/1031] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:23:17 2022] POST /identity/v3/users => generated 285 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:23:17.554929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650913b5-7d83-4d77-86e0-d0a868c85c2d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:17.624152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c0400-6314-4935-804b-fc92d303c1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:17.625372 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c0400-6314-4935-804b-fc92d303c1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:17.625904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c0400-6314-4935-804b-fc92d303c1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:17.626422 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c0400-6314-4935-804b-fc92d303c1c6 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:17.628228 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c0400-6314-4935-804b-fc92d303c1c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:17.628809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c0400-6314-4935-804b-fc92d303c1c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:17.631921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c0400-6314-4935-804b-fc92d303c1c6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:17.669178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 516/1032] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jun 7 08:23:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:17.672905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72aaab-f8c4-4343-a069-1501b5587055 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:17.780382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:17.810328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72aaab-f8c4-4343-a069-1501b5587055 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:17.862217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777301ca-552b-4738-a704-59079b2f8fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:17.863107 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777301ca-552b-4738-a704-59079b2f8fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:17.863400 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777301ca-552b-4738-a704-59079b2f8fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:17.863673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777301ca-552b-4738-a704-59079b2f8fc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:17.952839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777301ca-552b-4738-a704-59079b2f8fc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:17.956088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777301ca-552b-4738-a704-59079b2f8fc0 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:18.018466 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c77ac-29ea-4ef8-9078-be8597087dd3 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:18.019136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c77ac-29ea-4ef8-9078-be8597087dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:18.019734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c77ac-29ea-4ef8-9078-be8597087dd3 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:18.022788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 517/1033] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:18.034611 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d9ee5-42c6-47e4-ae8e-eb2afb7df565 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:18.035043 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d9ee5-42c6-47e4-ae8e-eb2afb7df565 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:18.035412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d9ee5-42c6-47e4-ae8e-eb2afb7df565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:18.057129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-df3d9ee5-42c6-47e4-ae8e-eb2afb7df565 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:18.059319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-df3d9ee5-42c6-47e4-ae8e-eb2afb7df565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:18.063539 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:18.097804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 517/1034] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:17 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:18.101965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777301ca-552b-4738-a704-59079b2f8fc0 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:18.176992 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 518/1035] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:23:18.210526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d9ee5-42c6-47e4-ae8e-eb2afb7df565 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:18.213992 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:18.228343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777301ca-552b-4738-a704-59079b2f8fc0 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:18.267984 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41835638-be4b-43e5-a8c8-67645e9877e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:18.268763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41835638-be4b-43e5-a8c8-67645e9877e6 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:18.269041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41835638-be4b-43e5-a8c8-67645e9877e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:18.269344 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41835638-be4b-43e5-a8c8-67645e9877e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:18.280481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41835638-be4b-43e5-a8c8-67645e9877e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:18.281985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41835638-be4b-43e5-a8c8-67645e9877e6 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-41835638-be4b-43e5-a8c8-67645e9877e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:18.304830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:18.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:18.309584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:18.309584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:18.309584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:18.309584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:18.309584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 518/1036] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:18.313922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41835638-be4b-43e5-a8c8-67645e9877e6 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:18.356582 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:18.388209 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d9ee5-42c6-47e4-ae8e-eb2afb7df565 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:18.428160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bc2ec-95a3-4334-83ab-ca587a0dddf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:18.428919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bc2ec-95a3-4334-83ab-ca587a0dddf5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:18.429247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bc2ec-95a3-4334-83ab-ca587a0dddf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:18.429423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bc2ec-95a3-4334-83ab-ca587a0dddf5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:18.430075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747bc2ec-95a3-4334-83ab-ca587a0dddf5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:18.431641 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747bc2ec-95a3-4334-83ab-ca587a0dddf5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-747bc2ec-95a3-4334-83ab-ca587a0dddf5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:18.444658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:18.445784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:18.446630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 519/1037] 10.222.0.39 () {56 vars in 1105 bytes} [Tue Jun 7 08:23:18 2022] GET /identity/v3/services/image => generated 86 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:18.457035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bc2ec-95a3-4334-83ab-ca587a0dddf5 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:18.470198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41835638-be4b-43e5-a8c8-67645e9877e6 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:18.512411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a0a82-09e2-4f1e-956a-f7c1dc524a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:18.513115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a0a82-09e2-4f1e-956a-f7c1dc524a0e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:18.513404 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a0a82-09e2-4f1e-956a-f7c1dc524a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:18.513682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a0a82-09e2-4f1e-956a-f7c1dc524a0e None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:18.514381 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a0a82-09e2-4f1e-956a-f7c1dc524a0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:18.514804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a0a82-09e2-4f1e-956a-f7c1dc524a0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:18.516319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a0a82-09e2-4f1e-956a-f7c1dc524a0e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:18.533377 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 519/1038] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:18.534981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a0a82-09e2-4f1e-956a-f7c1dc524a0e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:18.570322 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:18.614889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bc2ec-95a3-4334-83ab-ca587a0dddf5 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:18.663692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784e0dc-fa68-44a0-b1b6-c11e78c5403f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:18.664743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784e0dc-fa68-44a0-b1b6-c11e78c5403f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:18.665025 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784e0dc-fa68-44a0-b1b6-c11e78c5403f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:18.665329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784e0dc-fa68-44a0-b1b6-c11e78c5403f None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:18.666459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784e0dc-fa68-44a0-b1b6-c11e78c5403f None admin] RBAC: Adding query filter params (name=image) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:18.666778 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784e0dc-fa68-44a0-b1b6-c11e78c5403f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:18.668255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784e0dc-fa68-44a0-b1b6-c11e78c5403f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:18.674306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 520/1039] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:18.679767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784e0dc-fa68-44a0-b1b6-c11e78c5403f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:18.689462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:18.707891 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a0a82-09e2-4f1e-956a-f7c1dc524a0e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:18.753975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef540646-eb7f-414c-908e-a29841b9a6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:18.755169 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef540646-eb7f-414c-908e-a29841b9a6fe None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:18.755728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef540646-eb7f-414c-908e-a29841b9a6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:18.756149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef540646-eb7f-414c-908e-a29841b9a6fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:18.766678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef540646-eb7f-414c-908e-a29841b9a6fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:18.768308 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef540646-eb7f-414c-908e-a29841b9a6fe None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-ef540646-eb7f-414c-908e-a29841b9a6fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:23:18.797278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:18.798525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:18.799441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:23:18.799441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:23:18.799441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:23:18.799441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:23:18.799441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:23:18.799441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:23:18.799441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:18.799441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:18.805000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 520/1040] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:18.805977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef540646-eb7f-414c-908e-a29841b9a6fe None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:18.812316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784e0dc-fa68-44a0-b1b6-c11e78c5403f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:18.861645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:18.868793 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e080598-bcfb-487e-b819-554bb58327eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:18.870162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e080598-bcfb-487e-b819-554bb58327eb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:18.870452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e080598-bcfb-487e-b819-554bb58327eb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:18.870733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e080598-bcfb-487e-b819-554bb58327eb None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:18.871678 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e080598-bcfb-487e-b819-554bb58327eb None admin] RBAC: Adding query filter params (type=image) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:18.872092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e080598-bcfb-487e-b819-554bb58327eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:18.874366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e080598-bcfb-487e-b819-554bb58327eb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:18.887844 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 521/1041] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:18.889440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e080598-bcfb-487e-b819-554bb58327eb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:18.957838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef540646-eb7f-414c-908e-a29841b9a6fe None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:19.010993 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fd229-42d2-450c-ac6f-869d962d75d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:19.011655 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fd229-42d2-450c-ac6f-869d962d75d0 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:19.011936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fd229-42d2-450c-ac6f-869d962d75d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:19.012206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fd229-42d2-450c-ac6f-869d962d75d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:19.013297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fd229-42d2-450c-ac6f-869d962d75d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:19.013297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fd229-42d2-450c-ac6f-869d962d75d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:19.014759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fd229-42d2-450c-ac6f-869d962d75d0 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:19.033438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 521/1042] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:23:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:19.034299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e080598-bcfb-487e-b819-554bb58327eb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:19.034452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fd229-42d2-450c-ac6f-869d962d75d0 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:19.052813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:19.081396 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57179781-1bb5-4f74-8c15-748909fa0e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:19.082050 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179781-1bb5-4f74-8c15-748909fa0e72 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:19.082321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179781-1bb5-4f74-8c15-748909fa0e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:19.082594 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179781-1bb5-4f74-8c15-748909fa0e72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:19.083827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57179781-1bb5-4f74-8c15-748909fa0e72 None admin] RBAC: Adding query filter params (interface=public, service_id=c9483a56ef5e4f58ac61145a52227ae8, region_id=RegionOne) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:19.084137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57179781-1bb5-4f74-8c15-748909fa0e72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:19.085886 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57179781-1bb5-4f74-8c15-748909fa0e72 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:19.092630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 522/1043] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jun 7 08:23:18 2022] GET /identity/v3/endpoints?service_id=c9483a56ef5e4f58ac61145a52227ae8&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:19.102591 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57179781-1bb5-4f74-8c15-748909fa0e72 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:19.163528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fd229-42d2-450c-ac6f-869d962d75d0 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:19.176694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:19.211858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630fcaa-4737-4426-a5c0-4041dc9b0e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:19.213342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630fcaa-4737-4426-a5c0-4041dc9b0e5b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:19.213723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630fcaa-4737-4426-a5c0-4041dc9b0e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:19.214057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630fcaa-4737-4426-a5c0-4041dc9b0e5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:19.215132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630fcaa-4737-4426-a5c0-4041dc9b0e5b None admin] RBAC: Adding query filter params (role.id=7db380ac539f4b79a45b254e5f57896d, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=3f2a2bf913d1412f996acf4401ba7673) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:19.215511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630fcaa-4737-4426-a5c0-4041dc9b0e5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:19.217256 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630fcaa-4737-4426-a5c0-4041dc9b0e5b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:19.225693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 522/1044] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:23:19 2022] GET /identity/v3/role_assignments?user.id=3f2a2bf913d1412f996acf4401ba7673&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=7db380ac539f4b79a45b254e5f57896d => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:19.243419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57179781-1bb5-4f74-8c15-748909fa0e72 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:19.294685 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f4f38-686e-4a17-b0a8-9883cd03331b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:19.295378 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f4f38-686e-4a17-b0a8-9883cd03331b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:19.295661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f4f38-686e-4a17-b0a8-9883cd03331b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:19.295934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f4f38-686e-4a17-b0a8-9883cd03331b None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:19.296643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f4f38-686e-4a17-b0a8-9883cd03331b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:19.298091 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f4f38-686e-4a17-b0a8-9883cd03331b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:19.306011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 523/1045] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jun 7 08:23:19 2022] GET /identity/v3/services => generated 1957 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:23:20.075628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921e554-85b4-4936-8de9-3ffe4dcc5446 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:20.076239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921e554-85b4-4936-8de9-3ffe4dcc5446 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:20.076239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921e554-85b4-4936-8de9-3ffe4dcc5446 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:20.077811 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 523/1046] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:20.083333 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78be2a-ef50-46de-9c46-7e998ae81913 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:20.084126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78be2a-ef50-46de-9c46-7e998ae81913 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:20.084126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78be2a-ef50-46de-9c46-7e998ae81913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:20.106075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2f78be2a-ef50-46de-9c46-7e998ae81913 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:20.108249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2f78be2a-ef50-46de-9c46-7e998ae81913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:20.218058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 524/1047] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:20.241887 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921e554-85b4-4936-8de9-3ffe4dcc5446 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:20.385721 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:20.428009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921e554-85b4-4936-8de9-3ffe4dcc5446 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:20.490924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb26fb-77e0-415a-9677-d08e7a26d58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:20.491632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb26fb-77e0-415a-9677-d08e7a26d58e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:20.491914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb26fb-77e0-415a-9677-d08e7a26d58e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:20.492189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb26fb-77e0-415a-9677-d08e7a26d58e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:20.505821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb26fb-77e0-415a-9677-d08e7a26d58e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-ecdb26fb-77e0-415a-9677-d08e7a26d58e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:20.514266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:20.515330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:20.521666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:20.521666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:20.521666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:20.521666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:20.521666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:20.521666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:20.521666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:20.521666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:20.521666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:20.521666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:20.521666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:20.521666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:20.521666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 524/1048] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:23:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:20.523848 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78be2a-ef50-46de-9c46-7e998ae81913 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:20.590932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:20.675247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78be2a-ef50-46de-9c46-7e998ae81913 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:20.715763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166061e0-5eee-4475-8452-e879924ad1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:20.716446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166061e0-5eee-4475-8452-e879924ad1cb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:20.716734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166061e0-5eee-4475-8452-e879924ad1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:20.717002 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166061e0-5eee-4475-8452-e879924ad1cb None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:20.717871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166061e0-5eee-4475-8452-e879924ad1cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:20.718180 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166061e0-5eee-4475-8452-e879924ad1cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:20.719538 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166061e0-5eee-4475-8452-e879924ad1cb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:20.733995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 525/1049] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:23:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:20.735863 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb26fb-77e0-415a-9677-d08e7a26d58e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:20.852612 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb26fb-77e0-415a-9677-d08e7a26d58e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:20.893265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d7189-a73b-4226-9eac-0407eb43d70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:20.896455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d7189-a73b-4226-9eac-0407eb43d70f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:20.896455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d7189-a73b-4226-9eac-0407eb43d70f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:20.896455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d7189-a73b-4226-9eac-0407eb43d70f None admin] PATH_INFO: `/v3/users/277353f2efb041459c86af2ff8cc6333` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:20.906665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:20.914742 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d7189-a73b-4226-9eac-0407eb43d70f None admin] RBAC: Authorizing `identity:get_user(user_id=277353f2efb041459c86af2ff8cc6333)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:20.916362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d7189-a73b-4226-9eac-0407eb43d70f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:20.918406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 525/1050] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:23:20 2022] GET /identity/v3/users/277353f2efb041459c86af2ff8cc6333 => generated 285 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:20.924084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166061e0-5eee-4475-8452-e879924ad1cb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:21.037752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166061e0-5eee-4475-8452-e879924ad1cb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:21.061432 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:21.087784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e3cac-7937-4270-99cd-e7efaeb24e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:21.088595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3cac-7937-4270-99cd-e7efaeb24e3e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:21.089017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3cac-7937-4270-99cd-e7efaeb24e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:21.089017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3cac-7937-4270-99cd-e7efaeb24e3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:21.162842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e3cac-7937-4270-99cd-e7efaeb24e3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:21.164305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e3cac-7937-4270-99cd-e7efaeb24e3e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:21.284800 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 526/1051] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:20 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:21.291391 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d7189-a73b-4226-9eac-0407eb43d70f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:21.405230 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d7189-a73b-4226-9eac-0407eb43d70f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:21.444271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdcca3-306c-42a0-9815-cedba0848746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:21.445027 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdcca3-306c-42a0-9815-cedba0848746 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:21.445351 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdcca3-306c-42a0-9815-cedba0848746 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:21.445630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdcca3-306c-42a0-9815-cedba0848746 None admin] PATH_INFO: `/v3/projects/96b655f8d0934bada5d6354560d488f2` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:21.446853 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdcca3-306c-42a0-9815-cedba0848746 None admin] RBAC: Authorizing `identity:get_project(project_id=96b655f8d0934bada5d6354560d488f2)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:21.448304 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdcca3-306c-42a0-9815-cedba0848746 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:21.450281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 526/1052] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:23:21 2022] GET /identity/v3/projects/96b655f8d0934bada5d6354560d488f2 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:21.456391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e3cac-7937-4270-99cd-e7efaeb24e3e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:21.472851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:21.562442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e3cac-7937-4270-99cd-e7efaeb24e3e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:21.599795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab9ff8-9000-4510-9ae0-9b555c55f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:21.600461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab9ff8-9000-4510-9ae0-9b555c55f154 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:21.600744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab9ff8-9000-4510-9ae0-9b555c55f154 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:21.601020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab9ff8-9000-4510-9ae0-9b555c55f154 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:21.601884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab9ff8-9000-4510-9ae0-9b555c55f154 None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=96b655f8d0934bada5d6354560d488f2, user.id=277353f2efb041459c86af2ff8cc6333) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:21.602197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab9ff8-9000-4510-9ae0-9b555c55f154 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:21.604361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab9ff8-9000-4510-9ae0-9b555c55f154 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:21.613011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 527/1053] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:23:21 2022] GET /identity/v3/role_assignments?user.id=277353f2efb041459c86af2ff8cc6333&scope.project.id=96b655f8d0934bada5d6354560d488f2&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:21.615086 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca95f5-8bbe-4c9d-bd1a-fb8121b0371d None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:21.615337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca95f5-8bbe-4c9d-bd1a-fb8121b0371d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:21.615519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca95f5-8bbe-4c9d-bd1a-fb8121b0371d None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:21.617255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 527/1054] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:21.624033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb44f0-4daa-440b-ae28-8e9bdce04fde None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:21.624363 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb44f0-4daa-440b-ae28-8e9bdce04fde None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:21.624630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb44f0-4daa-440b-ae28-8e9bdce04fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:21.644056 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-10bb44f0-4daa-440b-ae28-8e9bdce04fde None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:21.645374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-10bb44f0-4daa-440b-ae28-8e9bdce04fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:21.724589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 528/1055] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:21.748268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb44f0-4daa-440b-ae28-8e9bdce04fde None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:21.748720 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b77510-c0e3-4d6c-b8fd-7acbcd20710d None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:21.749004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b77510-c0e3-4d6c-b8fd-7acbcd20710d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:21.749305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b77510-c0e3-4d6c-b8fd-7acbcd20710d None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:21.750855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 528/1056] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:21.756456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c789e65-bc9a-415e-92f0-297dd05b00bd None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:21.756753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c789e65-bc9a-415e-92f0-297dd05b00bd None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:21.757009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c789e65-bc9a-415e-92f0-297dd05b00bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:21.760741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:21.775213 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-4c789e65-bc9a-415e-92f0-297dd05b00bd None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:21.776445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4c789e65-bc9a-415e-92f0-297dd05b00bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:21.844630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:21.855534 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 529/1057] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:23:21.873884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb44f0-4daa-440b-ae28-8e9bdce04fde None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:21.878908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c789e65-bc9a-415e-92f0-297dd05b00bd None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:21.917545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0600486-9ded-4533-a2d1-73484daf726d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:21.918465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0600486-9ded-4533-a2d1-73484daf726d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:21.918753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0600486-9ded-4533-a2d1-73484daf726d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:21.919031 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0600486-9ded-4533-a2d1-73484daf726d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:21.930048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0600486-9ded-4533-a2d1-73484daf726d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f0600486-9ded-4533-a2d1-73484daf726d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:21.938626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:21.939560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:21.940401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:21.940401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:21.940401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:21.940401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:21.940401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:21.940401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:21.940401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:21.940401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:21.940401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:21.940401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:21.940401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:23:21.940401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:21.940401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 529/1058] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Jun 7 08:23:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:21.945994 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0600486-9ded-4533-a2d1-73484daf726d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:22.002515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c789e65-bc9a-415e-92f0-297dd05b00bd None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:22.020797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:22.042412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fceb6b-c213-40f0-be8f-6db2a5277551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:22.043200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fceb6b-c213-40f0-be8f-6db2a5277551 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:22.043478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fceb6b-c213-40f0-be8f-6db2a5277551 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:22.043749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fceb6b-c213-40f0-be8f-6db2a5277551 None admin] PATH_INFO: `/v3/services/image` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:22.044503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fceb6b-c213-40f0-be8f-6db2a5277551 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:22.045914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fceb6b-c213-40f0-be8f-6db2a5277551 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:22.057073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0600486-9ded-4533-a2d1-73484daf726d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-32fceb6b-c213-40f0-be8f-6db2a5277551 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:22.058989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:22.060310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:22.061292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 530/1059] 10.222.0.39 () {56 vars in 1105 bytes} [Tue Jun 7 08:23:21 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:22.066141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fceb6b-c213-40f0-be8f-6db2a5277551 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:22.099055 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb654e7-8db3-4ed8-9122-b3d28b88cab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:22.099812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb654e7-8db3-4ed8-9122-b3d28b88cab9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:22.100019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb654e7-8db3-4ed8-9122-b3d28b88cab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:22.100294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb654e7-8db3-4ed8-9122-b3d28b88cab9 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:22.101041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb654e7-8db3-4ed8-9122-b3d28b88cab9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:22.101379 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb654e7-8db3-4ed8-9122-b3d28b88cab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:22.102740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb654e7-8db3-4ed8-9122-b3d28b88cab9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:22.113132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 530/1060] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jun 7 08:23:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:22.118664 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb654e7-8db3-4ed8-9122-b3d28b88cab9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:22.134072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:22.181034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fceb6b-c213-40f0-be8f-6db2a5277551 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:22.218901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86112819-ae47-4067-8785-cb34a976d6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:22.219565 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86112819-ae47-4067-8785-cb34a976d6af None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:22.219847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86112819-ae47-4067-8785-cb34a976d6af None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:22.220116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86112819-ae47-4067-8785-cb34a976d6af None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:22.220810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86112819-ae47-4067-8785-cb34a976d6af None admin] RBAC: Adding query filter params (name=image) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:22.221132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86112819-ae47-4067-8785-cb34a976d6af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:22.222489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86112819-ae47-4067-8785-cb34a976d6af None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:22.227247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb654e7-8db3-4ed8-9122-b3d28b88cab9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:22.227859 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 531/1061] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:22.234027 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86112819-ae47-4067-8785-cb34a976d6af None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:22.262851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:22.269497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b1482-c524-4b69-8bb6-ca0e937c5802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:22.270382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b1482-c524-4b69-8bb6-ca0e937c5802 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:22.270733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b1482-c524-4b69-8bb6-ca0e937c5802 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:22.271072 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b1482-c524-4b69-8bb6-ca0e937c5802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:22.283199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b1482-c524-4b69-8bb6-ca0e937c5802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:22.284922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b1482-c524-4b69-8bb6-ca0e937c5802 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-736b1482-c524-4b69-8bb6-ca0e937c5802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:22.303217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:22.304360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:22.305577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:22.305577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:22.305577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:22.305577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:22.309527 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 531/1062] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:22.310384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b1482-c524-4b69-8bb6-ca0e937c5802 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:22.346304 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86112819-ae47-4067-8785-cb34a976d6af None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:22.371014 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:22.383327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0c51b-d4ea-4e1c-9c59-6ece2eb2b8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:22.383999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0c51b-d4ea-4e1c-9c59-6ece2eb2b8fc None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:22.384284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0c51b-d4ea-4e1c-9c59-6ece2eb2b8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:22.384642 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0c51b-d4ea-4e1c-9c59-6ece2eb2b8fc None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:22.385389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0c51b-d4ea-4e1c-9c59-6ece2eb2b8fc None admin] RBAC: Adding query filter params (type=image) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:22.385812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0c51b-d4ea-4e1c-9c59-6ece2eb2b8fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:22.387203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0c51b-d4ea-4e1c-9c59-6ece2eb2b8fc None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:22.392421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 532/1063] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:22.397913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0c51b-d4ea-4e1c-9c59-6ece2eb2b8fc None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:22.425449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b1482-c524-4b69-8bb6-ca0e937c5802 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:22.460990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda7ca2-3131-40ad-bb99-4a14ebfe38a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:22.462411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda7ca2-3131-40ad-bb99-4a14ebfe38a7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:22.462411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda7ca2-3131-40ad-bb99-4a14ebfe38a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:22.462411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda7ca2-3131-40ad-bb99-4a14ebfe38a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:22.464241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda7ca2-3131-40ad-bb99-4a14ebfe38a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:22.464605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda7ca2-3131-40ad-bb99-4a14ebfe38a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:22.465951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda7ca2-3131-40ad-bb99-4a14ebfe38a7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:22.479554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 532/1064] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:22.486260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda7ca2-3131-40ad-bb99-4a14ebfe38a7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:22.514027 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0c51b-d4ea-4e1c-9c59-6ece2eb2b8fc None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:22.532567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:22.556064 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1f196-c4fb-4c63-b0c9-89522f7d4e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:22.557441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f196-c4fb-4c63-b0c9-89522f7d4e41 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:22.557894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f196-c4fb-4c63-b0c9-89522f7d4e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:22.558425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f196-c4fb-4c63-b0c9-89522f7d4e41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:22.559185 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1f196-c4fb-4c63-b0c9-89522f7d4e41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:22.560653 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1f196-c4fb-4c63-b0c9-89522f7d4e41 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:22.581000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 533/1065] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jun 7 08:23:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:23:22.594913 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda7ca2-3131-40ad-bb99-4a14ebfe38a7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:22.641520 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15556194-7b95-4189-82fb-44ef625445ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:22.642344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15556194-7b95-4189-82fb-44ef625445ca None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:22.642628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15556194-7b95-4189-82fb-44ef625445ca None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:22.642898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15556194-7b95-4189-82fb-44ef625445ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:22.652137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15556194-7b95-4189-82fb-44ef625445ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:22.653706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15556194-7b95-4189-82fb-44ef625445ca None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-15556194-7b95-4189-82fb-44ef625445ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:23:22.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:22.670933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:22.672619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:22.672619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:22.672619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:23:22.672619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:23:22.672619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:23:22.672619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:23:22.672619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:23:22.672619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:23:22.672619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:23:22.672619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:22.672619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 533/1066] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jun 7 08:23:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:22.676832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1f196-c4fb-4c63-b0c9-89522f7d4e41 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:22.727469 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:22.798865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1f196-c4fb-4c63-b0c9-89522f7d4e41 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:22.837909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844714e-ae40-4bdc-bf2b-dad0ee9915ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:22.838617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844714e-ae40-4bdc-bf2b-dad0ee9915ab None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:22.839127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844714e-ae40-4bdc-bf2b-dad0ee9915ab None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:22.839405 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844714e-ae40-4bdc-bf2b-dad0ee9915ab None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:22.840413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844714e-ae40-4bdc-bf2b-dad0ee9915ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:22.840616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844714e-ae40-4bdc-bf2b-dad0ee9915ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:22.842075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844714e-ae40-4bdc-bf2b-dad0ee9915ab None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:22.865116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 534/1067] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jun 7 08:23:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:22.867898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15556194-7b95-4189-82fb-44ef625445ca None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:22.983252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:22.988345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15556194-7b95-4189-82fb-44ef625445ca None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:23.025984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37f851-21f1-4446-ae0b-511b4ab47967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:23.026840 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37f851-21f1-4446-ae0b-511b4ab47967 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:23.027082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37f851-21f1-4446-ae0b-511b4ab47967 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:23.027408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37f851-21f1-4446-ae0b-511b4ab47967 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:23.037724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37f851-21f1-4446-ae0b-511b4ab47967 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:23.039160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37f851-21f1-4446-ae0b-511b4ab47967 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-db37f851-21f1-4446-ae0b-511b4ab47967 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:23.057472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:23.058391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:23.059453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:23.059453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:23.059453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:23.059453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:23.059453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 534/1068] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:23.064235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844714e-ae40-4bdc-bf2b-dad0ee9915ab None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:23.169935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:23.173966 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844714e-ae40-4bdc-bf2b-dad0ee9915ab None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:23.219692 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ff130-7f03-441c-bd9f-16198899c1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:23.220364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ff130-7f03-441c-bd9f-16198899c1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:23.220652 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ff130-7f03-441c-bd9f-16198899c1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:23.220924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ff130-7f03-441c-bd9f-16198899c1e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:23.221693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ff130-7f03-441c-bd9f-16198899c1e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:23.222003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ff130-7f03-441c-bd9f-16198899c1e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:23.223390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ff130-7f03-441c-bd9f-16198899c1e8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:23.242163 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 535/1069] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:23.243406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37f851-21f1-4446-ae0b-511b4ab47967 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:23.364807 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37f851-21f1-4446-ae0b-511b4ab47967 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:23.409613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a222c41-e349-43a6-b51a-0b74d12c76a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:23.411400 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a222c41-e349-43a6-b51a-0b74d12c76a3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:23.411690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a222c41-e349-43a6-b51a-0b74d12c76a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:23.411974 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a222c41-e349-43a6-b51a-0b74d12c76a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:23.421279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a222c41-e349-43a6-b51a-0b74d12c76a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:23.422736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a222c41-e349-43a6-b51a-0b74d12c76a3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-6a222c41-e349-43a6-b51a-0b74d12c76a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:23:23.437858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:23.438846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:23.439792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:23:23.439792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:23:23.439792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:23:23.439792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:23:23.439792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:23:23.439792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:23:23.439792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:23.439792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:23.439792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 535/1070] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:23.446241 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ff130-7f03-441c-bd9f-16198899c1e8 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:23.567679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ff130-7f03-441c-bd9f-16198899c1e8 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:23.606540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f55d18-05ed-41bd-8aa8-b3d9177d0fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:23.607237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f55d18-05ed-41bd-8aa8-b3d9177d0fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:23.607519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f55d18-05ed-41bd-8aa8-b3d9177d0fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:23.608035 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f55d18-05ed-41bd-8aa8-b3d9177d0fe7 None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:23.608609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f55d18-05ed-41bd-8aa8-b3d9177d0fe7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:23.608922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f55d18-05ed-41bd-8aa8-b3d9177d0fe7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:23.610476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f55d18-05ed-41bd-8aa8-b3d9177d0fe7 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:23.626775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 536/1071] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:23:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:23.632936 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a222c41-e349-43a6-b51a-0b74d12c76a3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:23.650852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:23.782868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a222c41-e349-43a6-b51a-0b74d12c76a3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:23.828988 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172969a-072d-4b3b-9f5c-62adfcd25f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:23.829922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172969a-072d-4b3b-9f5c-62adfcd25f50 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:23.830209 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172969a-072d-4b3b-9f5c-62adfcd25f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:23.830488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172969a-072d-4b3b-9f5c-62adfcd25f50 None admin] PATH_INFO: `/v3/projects/07b861f95bf3498d990785ba72474d82/users/3f2a2bf913d1412f996acf4401ba7673/roles/7db380ac539f4b79a45b254e5f57896d` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:23.834607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172969a-072d-4b3b-9f5c-62adfcd25f50 None admin] RBAC: Authorizing `identity:create_grant(project_id=07b861f95bf3498d990785ba72474d82, user_id=3f2a2bf913d1412f996acf4401ba7673, role_id=7db380ac539f4b79a45b254e5f57896d)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:23.838264 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172969a-072d-4b3b-9f5c-62adfcd25f50 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:23.855408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 536/1072] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:23:23 2022] PUT /identity/v3/projects/07b861f95bf3498d990785ba72474d82/users/3f2a2bf913d1412f996acf4401ba7673/roles/7db380ac539f4b79a45b254e5f57896d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:23:24.001302 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14503a-bc5b-4228-bc7d-4b2803fb715c None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:24.001808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14503a-bc5b-4228-bc7d-4b2803fb715c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:24.002328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14503a-bc5b-4228-bc7d-4b2803fb715c None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:24.004182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 537/1073] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:24.010375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def00455-19d3-4c36-a8d3-61df7bf89744 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:24.010797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def00455-19d3-4c36-a8d3-61df7bf89744 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:24.011069 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def00455-19d3-4c36-a8d3-61df7bf89744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:24.032168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-def00455-19d3-4c36-a8d3-61df7bf89744 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:24.033529 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-def00455-19d3-4c36-a8d3-61df7bf89744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:24.123695 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:24.133427 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 537/1074] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:24.154424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14503a-bc5b-4228-bc7d-4b2803fb715c None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:24.182844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:24.298647 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14503a-bc5b-4228-bc7d-4b2803fb715c None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:24.341670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e6c4b-2fa2-437c-a917-497fb02c4527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:24.342474 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e6c4b-2fa2-437c-a917-497fb02c4527 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:24.342844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e6c4b-2fa2-437c-a917-497fb02c4527 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:24.343123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e6c4b-2fa2-437c-a917-497fb02c4527 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:24.356165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e6c4b-2fa2-437c-a917-497fb02c4527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-740e6c4b-2fa2-437c-a917-497fb02c4527 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:24.364323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:24.366309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:24.367960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:24.367960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:24.367960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:24.367960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:24.367960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:24.367960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:24.367960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:24.367960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:24.367960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:24.367960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:24.367960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:24.367960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:24.381197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 538/1075] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:23:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:24.384093 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def00455-19d3-4c36-a8d3-61df7bf89744 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:24.500141 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def00455-19d3-4c36-a8d3-61df7bf89744 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:24.542934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50a60c-de4c-4ae5-a1e6-c212031eca25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:24.543915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50a60c-de4c-4ae5-a1e6-c212031eca25 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:24.544409 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50a60c-de4c-4ae5-a1e6-c212031eca25 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:24.544863 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50a60c-de4c-4ae5-a1e6-c212031eca25 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:24.545883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50a60c-de4c-4ae5-a1e6-c212031eca25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:24.546365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50a60c-de4c-4ae5-a1e6-c212031eca25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:24.547930 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50a60c-de4c-4ae5-a1e6-c212031eca25 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:24.557989 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 538/1076] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:23:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:24.564013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e6c4b-2fa2-437c-a917-497fb02c4527 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:24.677219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:24.702346 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e6c4b-2fa2-437c-a917-497fb02c4527 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:24.741544 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81697ce-a6bb-4a88-9b9c-74bf0e3eb82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:24.742493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81697ce-a6bb-4a88-9b9c-74bf0e3eb82b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:24.742777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81697ce-a6bb-4a88-9b9c-74bf0e3eb82b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:24.743046 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81697ce-a6bb-4a88-9b9c-74bf0e3eb82b None admin] PATH_INFO: `/v3/users/277353f2efb041459c86af2ff8cc6333` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:24.744814 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81697ce-a6bb-4a88-9b9c-74bf0e3eb82b None admin] RBAC: Authorizing `identity:get_user(user_id=277353f2efb041459c86af2ff8cc6333)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:24.746299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81697ce-a6bb-4a88-9b9c-74bf0e3eb82b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:24.752725 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 539/1077] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:23:24 2022] GET /identity/v3/users/277353f2efb041459c86af2ff8cc6333 => generated 285 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:24.754132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50a60c-de4c-4ae5-a1e6-c212031eca25 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:24.843385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:24.896598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50a60c-de4c-4ae5-a1e6-c212031eca25 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:24.933818 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c58158-d58e-4412-9da5-cc47afdd0a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:24.934688 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58158-d58e-4412-9da5-cc47afdd0a8f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:24.934967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58158-d58e-4412-9da5-cc47afdd0a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:24.935239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58158-d58e-4412-9da5-cc47afdd0a8f None admin] PATH_INFO: `/v3/projects/96b655f8d0934bada5d6354560d488f2` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:24.936441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c58158-d58e-4412-9da5-cc47afdd0a8f None admin] RBAC: Authorizing `identity:get_project(project_id=96b655f8d0934bada5d6354560d488f2)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:24.937900 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c58158-d58e-4412-9da5-cc47afdd0a8f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:24.940730 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 539/1078] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:23:24 2022] GET /identity/v3/projects/96b655f8d0934bada5d6354560d488f2 => generated 311 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:24.945689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81697ce-a6bb-4a88-9b9c-74bf0e3eb82b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:25.058819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7519fa-9f3b-40fa-a6bd-d5cc425e4333 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:25.059019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7519fa-9f3b-40fa-a6bd-d5cc425e4333 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:25.059284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7519fa-9f3b-40fa-a6bd-d5cc425e4333 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:25.061164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 540/1079] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:23:25.067862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9786d1-2361-42c3-99a8-a073539bf14c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:25.068144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9786d1-2361-42c3-99a8-a073539bf14c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:25.068451 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9786d1-2361-42c3-99a8-a073539bf14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:25.077607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81697ce-a6bb-4a88-9b9c-74bf0e3eb82b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:25.089175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3c9786d1-2361-42c3-99a8-a073539bf14c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:25.090404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3c9786d1-2361-42c3-99a8-a073539bf14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:25.118102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c03f8-ffc4-4284-b205-670d0fca715b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:25.119550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c03f8-ffc4-4284-b205-670d0fca715b None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:25.119869 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c03f8-ffc4-4284-b205-670d0fca715b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:25.120146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c03f8-ffc4-4284-b205-670d0fca715b None admin] PATH_INFO: `/v3/projects/96b655f8d0934bada5d6354560d488f2/users/277353f2efb041459c86af2ff8cc6333/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:25.122578 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c03f8-ffc4-4284-b205-670d0fca715b None admin] RBAC: Authorizing `identity:create_grant(project_id=96b655f8d0934bada5d6354560d488f2, user_id=277353f2efb041459c86af2ff8cc6333, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:25.124187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c03f8-ffc4-4284-b205-670d0fca715b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:25.149184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 540/1080] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:23:24 2022] PUT /identity/v3/projects/96b655f8d0934bada5d6354560d488f2/users/277353f2efb041459c86af2ff8cc6333/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:23:25.204671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 541/1081] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:25.228647 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c03f8-ffc4-4284-b205-670d0fca715b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:25.242217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:25.367935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c03f8-ffc4-4284-b205-670d0fca715b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:25.411815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0d0a6-5d3c-45d3-a2d8-00e372cf6b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:25.412469 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0d0a6-5d3c-45d3-a2d8-00e372cf6b98 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:25.412752 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0d0a6-5d3c-45d3-a2d8-00e372cf6b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:25.413026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0d0a6-5d3c-45d3-a2d8-00e372cf6b98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:25.500590 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0d0a6-5d3c-45d3-a2d8-00e372cf6b98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:25.502085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0d0a6-5d3c-45d3-a2d8-00e372cf6b98 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:25.513301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:25.613444 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 541/1082] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:25.619986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9786d1-2361-42c3-99a8-a073539bf14c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:25.694468 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:25.740364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9786d1-2361-42c3-99a8-a073539bf14c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:25.777625 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b400d9-4ab9-4b92-afdf-a82b74787081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:25.778436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b400d9-4ab9-4b92-afdf-a82b74787081 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:25.778720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b400d9-4ab9-4b92-afdf-a82b74787081 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:25.778993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b400d9-4ab9-4b92-afdf-a82b74787081 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:25.809895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b400d9-4ab9-4b92-afdf-a82b74787081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:25.811379 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b400d9-4ab9-4b92-afdf-a82b74787081 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-85b400d9-4ab9-4b92-afdf-a82b74787081 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:25.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:25.834252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:25.835721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:25.835721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:25.835721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:25.835721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:25.837569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 542/1083] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:25.840402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0d0a6-5d3c-45d3-a2d8-00e372cf6b98 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:25.972273 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0d0a6-5d3c-45d3-a2d8-00e372cf6b98 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:25.983722 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:26.016329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0ba45-138c-4e15-9b5f-1cc2c3bfaac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:26.017019 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0ba45-138c-4e15-9b5f-1cc2c3bfaac5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:26.017337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0ba45-138c-4e15-9b5f-1cc2c3bfaac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:26.017622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0ba45-138c-4e15-9b5f-1cc2c3bfaac5 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:26.018355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0ba45-138c-4e15-9b5f-1cc2c3bfaac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:26.018668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0ba45-138c-4e15-9b5f-1cc2c3bfaac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:26.020057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0ba45-138c-4e15-9b5f-1cc2c3bfaac5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:26.041189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 542/1084] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:26.272313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba45eca-cd03-4895-9d92-632aa8a30eb0 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:26.272504 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba45eca-cd03-4895-9d92-632aa8a30eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:26.272785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba45eca-cd03-4895-9d92-632aa8a30eb0 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:26.274218 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 543/1085] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:23:26.280408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2355819-6872-435c-813c-a6b897de680d None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:26.280721 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2355819-6872-435c-813c-a6b897de680d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:26.280983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2355819-6872-435c-813c-a6b897de680d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:26.303222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-a2355819-6872-435c-813c-a6b897de680d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:26.304431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a2355819-6872-435c-813c-a6b897de680d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:26.400126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 543/1086] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:26.424561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba45eca-cd03-4895-9d92-632aa8a30eb0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:26.536928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba45eca-cd03-4895-9d92-632aa8a30eb0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:26.552265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:26.575081 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602207e-50f0-44c3-b209-1a1d62fe35d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:26.575931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602207e-50f0-44c3-b209-1a1d62fe35d0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:26.576183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602207e-50f0-44c3-b209-1a1d62fe35d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:26.576460 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602207e-50f0-44c3-b209-1a1d62fe35d0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:26.587038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602207e-50f0-44c3-b209-1a1d62fe35d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-d602207e-50f0-44c3-b209-1a1d62fe35d0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:26.595291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:26.596261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:26.597152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:26.597152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:26.597152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:26.597152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:26.597152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:26.597152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:26.597152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:26.597152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:26.597152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:26.597152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:26.597152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:23:26.597152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:26.597152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 544/1087] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Jun 7 08:23:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:26.602521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2355819-6872-435c-813c-a6b897de680d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:26.700674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:26.704960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2355819-6872-435c-813c-a6b897de680d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:26.740263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d3e2f-3bd4-4e70-b70e-7558e1a57b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:26.740935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d3e2f-3bd4-4e70-b70e-7558e1a57b15 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:26.741259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d3e2f-3bd4-4e70-b70e-7558e1a57b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:26.741540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d3e2f-3bd4-4e70-b70e-7558e1a57b15 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:26.742285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d3e2f-3bd4-4e70-b70e-7558e1a57b15 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:26.742593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d3e2f-3bd4-4e70-b70e-7558e1a57b15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:26.744033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d3e2f-3bd4-4e70-b70e-7558e1a57b15 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:26.752231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 544/1088] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jun 7 08:23:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:26.757676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602207e-50f0-44c3-b209-1a1d62fe35d0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:26.862229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602207e-50f0-44c3-b209-1a1d62fe35d0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:26.903929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588830b-9d85-4cf0-ad2c-e9c08ed96298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:26.904731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588830b-9d85-4cf0-ad2c-e9c08ed96298 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:26.905013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588830b-9d85-4cf0-ad2c-e9c08ed96298 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:26.905308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588830b-9d85-4cf0-ad2c-e9c08ed96298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:26.916891 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588830b-9d85-4cf0-ad2c-e9c08ed96298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:26.918569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588830b-9d85-4cf0-ad2c-e9c08ed96298 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-5588830b-9d85-4cf0-ad2c-e9c08ed96298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:26.935770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:26.936796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:26.937736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:26.937736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:26.937736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:26.937736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:26.937736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 545/1089] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:26.942794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d3e2f-3bd4-4e70-b70e-7558e1a57b15 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:27.045430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d3e2f-3bd4-4e70-b70e-7558e1a57b15 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:27.080031 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568a590-4fc7-4f47-ba86-d10c0ded8bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:27.080697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568a590-4fc7-4f47-ba86-d10c0ded8bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:27.080982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568a590-4fc7-4f47-ba86-d10c0ded8bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:27.081287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568a590-4fc7-4f47-ba86-d10c0ded8bd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:27.082012 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568a590-4fc7-4f47-ba86-d10c0ded8bd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:27.082319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568a590-4fc7-4f47-ba86-d10c0ded8bd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:27.083708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568a590-4fc7-4f47-ba86-d10c0ded8bd6 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:27.096753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 545/1090] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:27.102316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588830b-9d85-4cf0-ad2c-e9c08ed96298 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:27.130060 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:27.207297 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588830b-9d85-4cf0-ad2c-e9c08ed96298 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:27.250681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9053e-8c65-46f3-b4a6-431446217933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:27.251505 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9053e-8c65-46f3-b4a6-431446217933 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:27.251888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9053e-8c65-46f3-b4a6-431446217933 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:27.252166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9053e-8c65-46f3-b4a6-431446217933 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:27.262597 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9053e-8c65-46f3-b4a6-431446217933 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:27.264220 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9053e-8c65-46f3-b4a6-431446217933 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-15e9053e-8c65-46f3-b4a6-431446217933 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:23:27.282115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:27.283316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:27.284257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:27.284257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:27.284257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:23:27.284257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:23:27.284257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:23:27.284257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:23:27.284257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:23:27.284257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:23:27.284257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:23:27.284257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:27.284257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 546/1091] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jun 7 08:23:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:27.289797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568a590-4fc7-4f47-ba86-d10c0ded8bd6 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:27.311991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:27.394525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568a590-4fc7-4f47-ba86-d10c0ded8bd6 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:27.432298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c08064-a959-4c15-b7e0-402951a86067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:27.433027 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c08064-a959-4c15-b7e0-402951a86067 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:27.433340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c08064-a959-4c15-b7e0-402951a86067 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:27.433628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c08064-a959-4c15-b7e0-402951a86067 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:27.434570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c08064-a959-4c15-b7e0-402951a86067 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:27.434881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c08064-a959-4c15-b7e0-402951a86067 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:27.436320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c08064-a959-4c15-b7e0-402951a86067 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:27.455977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 546/1092] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jun 7 08:23:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:27.461498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9053e-8c65-46f3-b4a6-431446217933 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:27.548749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:27.567141 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9053e-8c65-46f3-b4a6-431446217933 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:27.602835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a74637-54e0-4e74-8205-4837b42fcb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:27.603523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a74637-54e0-4e74-8205-4837b42fcb60 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:27.603808 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a74637-54e0-4e74-8205-4837b42fcb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:27.604086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a74637-54e0-4e74-8205-4837b42fcb60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:27.617608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547282dd-bdf3-4e29-9bce-8ba610780f4f None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:27.617903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547282dd-bdf3-4e29-9bce-8ba610780f4f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:27.618183 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547282dd-bdf3-4e29-9bce-8ba610780f4f None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:27.620364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 547/1093] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:23:27.625611 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca14db-cd1e-4f78-8685-298573f39c05 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:27.625897 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca14db-cd1e-4f78-8685-298573f39c05 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:27.626172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca14db-cd1e-4f78-8685-298573f39c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:27.643647 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-9fca14db-cd1e-4f78-8685-298573f39c05 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:27.644945 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9fca14db-cd1e-4f78-8685-298573f39c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:27.675394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a74637-54e0-4e74-8205-4837b42fcb60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:27.678052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a74637-54e0-4e74-8205-4837b42fcb60 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:27.720398 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 548/1094] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:23:27.744342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca14db-cd1e-4f78-8685-298573f39c05 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:27.754444 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:27.771065 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:27.788803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 547/1095] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:27.794787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a74637-54e0-4e74-8205-4837b42fcb60 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:27.846264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca14db-cd1e-4f78-8685-298573f39c05 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:27.882173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eece2cf-47ef-464e-94cf-a179ea5ff971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:27.882961 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eece2cf-47ef-464e-94cf-a179ea5ff971 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:27.883236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eece2cf-47ef-464e-94cf-a179ea5ff971 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:27.883511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eece2cf-47ef-464e-94cf-a179ea5ff971 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:27.899513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a74637-54e0-4e74-8205-4837b42fcb60 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:27.899639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eece2cf-47ef-464e-94cf-a179ea5ff971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-5eece2cf-47ef-464e-94cf-a179ea5ff971 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:27.907616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:27.908626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:27.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:27.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:27.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:27.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:27.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:27.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:27.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:27.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:27.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:27.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:27.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:27.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:27.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 549/1096] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:23:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:27.914732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eece2cf-47ef-464e-94cf-a179ea5ff971 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:27.936901 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a5e0f-e4e8-4db5-a2a2-5bca00126189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:27.937730 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a5e0f-e4e8-4db5-a2a2-5bca00126189 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:27.938015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a5e0f-e4e8-4db5-a2a2-5bca00126189 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:27.938292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a5e0f-e4e8-4db5-a2a2-5bca00126189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:27.948770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a5e0f-e4e8-4db5-a2a2-5bca00126189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:27.950245 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a5e0f-e4e8-4db5-a2a2-5bca00126189 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-ba4a5e0f-e4e8-4db5-a2a2-5bca00126189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:27.968213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:27.969272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:27.970206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:27.970206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:27.970206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:27.970206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:27.970206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 548/1097] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:27.975217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a5e0f-e4e8-4db5-a2a2-5bca00126189 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:27.985179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:28.013793 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:28.025299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eece2cf-47ef-464e-94cf-a179ea5ff971 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:28.062528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b870721-7c23-4fce-9cd0-b5e66826d105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:28.063194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870721-7c23-4fce-9cd0-b5e66826d105 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:28.063484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870721-7c23-4fce-9cd0-b5e66826d105 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:28.063753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870721-7c23-4fce-9cd0-b5e66826d105 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:28.064493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b870721-7c23-4fce-9cd0-b5e66826d105 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:28.064794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b870721-7c23-4fce-9cd0-b5e66826d105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:28.066218 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b870721-7c23-4fce-9cd0-b5e66826d105 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:28.076207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 550/1098] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:23:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:28.082238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b870721-7c23-4fce-9cd0-b5e66826d105 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:28.083033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a5e0f-e4e8-4db5-a2a2-5bca00126189 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:28.119711 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06713583-9559-4347-b8ef-f89b6e57a139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:28.120374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06713583-9559-4347-b8ef-f89b6e57a139 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:28.120658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06713583-9559-4347-b8ef-f89b6e57a139 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:28.120937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06713583-9559-4347-b8ef-f89b6e57a139 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:28.121704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06713583-9559-4347-b8ef-f89b6e57a139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:28.122020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06713583-9559-4347-b8ef-f89b6e57a139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:28.123395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06713583-9559-4347-b8ef-f89b6e57a139 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:28.134925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 549/1099] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:28.140715 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06713583-9559-4347-b8ef-f89b6e57a139 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:28.185714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b870721-7c23-4fce-9cd0-b5e66826d105 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:28.215792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:28.222603 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d122ba5-c054-40b0-9b8b-4a00e55181f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:28.223482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d122ba5-c054-40b0-9b8b-4a00e55181f3 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:28.223766 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d122ba5-c054-40b0-9b8b-4a00e55181f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:28.224044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d122ba5-c054-40b0-9b8b-4a00e55181f3 None admin] PATH_INFO: `/v3/users/277353f2efb041459c86af2ff8cc6333` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:28.225307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d122ba5-c054-40b0-9b8b-4a00e55181f3 None admin] RBAC: Authorizing `identity:get_user(user_id=277353f2efb041459c86af2ff8cc6333)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:28.226770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d122ba5-c054-40b0-9b8b-4a00e55181f3 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:28.229018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 551/1100] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:23:28 2022] GET /identity/v3/users/277353f2efb041459c86af2ff8cc6333 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:28.234675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d122ba5-c054-40b0-9b8b-4a00e55181f3 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:28.239805 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:28.246799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06713583-9559-4347-b8ef-f89b6e57a139 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:28.281234 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa9bde-84e9-4f03-baa1-e3c878ac9973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:28.282328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa9bde-84e9-4f03-baa1-e3c878ac9973 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:28.282648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa9bde-84e9-4f03-baa1-e3c878ac9973 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:28.282942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa9bde-84e9-4f03-baa1-e3c878ac9973 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:28.291116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa9bde-84e9-4f03-baa1-e3c878ac9973 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:28.292584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa9bde-84e9-4f03-baa1-e3c878ac9973 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-e2aa9bde-84e9-4f03-baa1-e3c878ac9973 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:23:28.307058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:28.308039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:28.309059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:23:28.309059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:23:28.309059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:23:28.309059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:23:28.309059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:23:28.309059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:23:28.309059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:28.309059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:28.309059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 550/1101] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:28.314171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa9bde-84e9-4f03-baa1-e3c878ac9973 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:28.336457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d122ba5-c054-40b0-9b8b-4a00e55181f3 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:28.371167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77158095-904e-4031-8df1-c27a5818e967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:28.371819 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77158095-904e-4031-8df1-c27a5818e967 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:28.372095 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77158095-904e-4031-8df1-c27a5818e967 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:28.372370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77158095-904e-4031-8df1-c27a5818e967 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:28.413725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa9bde-84e9-4f03-baa1-e3c878ac9973 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:28.438858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:28.442839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77158095-904e-4031-8df1-c27a5818e967 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:28.444290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77158095-904e-4031-8df1-c27a5818e967 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:28.450067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269de0f-c05f-4c9d-8d86-8fcc631bef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:28.450726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269de0f-c05f-4c9d-8d86-8fcc631bef03 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:28.451006 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269de0f-c05f-4c9d-8d86-8fcc631bef03 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:28.451276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269de0f-c05f-4c9d-8d86-8fcc631bef03 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:28.452062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269de0f-c05f-4c9d-8d86-8fcc631bef03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:28.452371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269de0f-c05f-4c9d-8d86-8fcc631bef03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:28.454067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269de0f-c05f-4c9d-8d86-8fcc631bef03 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:28.466479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 551/1102] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:23:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:28.472872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269de0f-c05f-4c9d-8d86-8fcc631bef03 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:28.492373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:28.550611 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 552/1103] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:28.552097 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77158095-904e-4031-8df1-c27a5818e967 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:28.578981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269de0f-c05f-4c9d-8d86-8fcc631bef03 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:28.635867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a565e72-4327-44be-9446-a47a47c1b676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:28.636543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a565e72-4327-44be-9446-a47a47c1b676 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:28.636825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a565e72-4327-44be-9446-a47a47c1b676 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:28.637125 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a565e72-4327-44be-9446-a47a47c1b676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:28.637962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a565e72-4327-44be-9446-a47a47c1b676 None admin] RBAC: Adding query filter params (role.id=7db380ac539f4b79a45b254e5f57896d, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=3f2a2bf913d1412f996acf4401ba7673) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:28.638273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a565e72-4327-44be-9446-a47a47c1b676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:28.639694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a565e72-4327-44be-9446-a47a47c1b676 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:28.647271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 552/1104] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:23:28 2022] GET /identity/v3/role_assignments?user.id=3f2a2bf913d1412f996acf4401ba7673&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=7db380ac539f4b79a45b254e5f57896d => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:28.678697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:28.684128 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77158095-904e-4031-8df1-c27a5818e967 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:28.722546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592bfdc-a311-4e99-ad75-deddc71d5a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:28.723339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592bfdc-a311-4e99-ad75-deddc71d5a8d None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:28.723620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592bfdc-a311-4e99-ad75-deddc71d5a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:28.723894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592bfdc-a311-4e99-ad75-deddc71d5a8d None admin] PATH_INFO: `/v3/projects/96b655f8d0934bada5d6354560d488f2` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:28.725149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592bfdc-a311-4e99-ad75-deddc71d5a8d None admin] RBAC: Authorizing `identity:get_project(project_id=96b655f8d0934bada5d6354560d488f2)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:28.726611 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592bfdc-a311-4e99-ad75-deddc71d5a8d None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:28.729142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 553/1105] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:23:28 2022] GET /identity/v3/projects/96b655f8d0934bada5d6354560d488f2 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:28.740251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a565e72-4327-44be-9446-a47a47c1b676 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:28.830641 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:28.852107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a565e72-4327-44be-9446-a47a47c1b676 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:28.893842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d71b2-6a22-46e9-899b-53dbfcbdd6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:28.894544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d71b2-6a22-46e9-899b-53dbfcbdd6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:28.894850 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d71b2-6a22-46e9-899b-53dbfcbdd6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:28.895127 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d71b2-6a22-46e9-899b-53dbfcbdd6a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:28.896012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d71b2-6a22-46e9-899b-53dbfcbdd6a6 None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=96b655f8d0934bada5d6354560d488f2, user.id=277353f2efb041459c86af2ff8cc6333) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:28.896321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d71b2-6a22-46e9-899b-53dbfcbdd6a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:28.897826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d71b2-6a22-46e9-899b-53dbfcbdd6a6 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:28.906365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 553/1106] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:23:28 2022] GET /identity/v3/role_assignments?user.id=277353f2efb041459c86af2ff8cc6333&scope.project.id=96b655f8d0934bada5d6354560d488f2&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:31.256797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf24cc5-863c-4791-b76f-67cc349cd66f None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:31.257413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf24cc5-863c-4791-b76f-67cc349cd66f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:31.257413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf24cc5-863c-4791-b76f-67cc349cd66f None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:31.258808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 554/1107] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:31.264178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8063-51d1-49f7-b93f-d3099dd85cb7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:31.264614 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8063-51d1-49f7-b93f-d3099dd85cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:31.264739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8063-51d1-49f7-b93f-d3099dd85cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:31.282103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4a2c8063-51d1-49f7-b93f-d3099dd85cb7 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:31.284217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4a2c8063-51d1-49f7-b93f-d3099dd85cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:31.351998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 554/1108] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:31.377609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf24cc5-863c-4791-b76f-67cc349cd66f None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:31.481388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf24cc5-863c-4791-b76f-67cc349cd66f None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:31.516710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651ba9e-c529-4896-a312-478095aefb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:31.517798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651ba9e-c529-4896-a312-478095aefb24 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:31.518321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651ba9e-c529-4896-a312-478095aefb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:31.518786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651ba9e-c529-4896-a312-478095aefb24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:31.520026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651ba9e-c529-4896-a312-478095aefb24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:31.521513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651ba9e-c529-4896-a312-478095aefb24 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:31.523163 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 555/1109] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:31.529782 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c8063-51d1-49f7-b93f-d3099dd85cb7 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:31.603854 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:31.628226 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c8063-51d1-49f7-b93f-d3099dd85cb7 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:31.662265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c424c-55ad-4f00-84da-cbaf03a28f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:31.662928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c424c-55ad-4f00-84da-cbaf03a28f62 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:31.663214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c424c-55ad-4f00-84da-cbaf03a28f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:31.663489 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c424c-55ad-4f00-84da-cbaf03a28f62 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:31.664189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c424c-55ad-4f00-84da-cbaf03a28f62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:31.665836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c424c-55ad-4f00-84da-cbaf03a28f62 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:31.685399 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 555/1110] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:23:31 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:23:34.024482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca89611-c3b8-4ddd-894a-ec77f1cdb389 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:34.025036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca89611-c3b8-4ddd-894a-ec77f1cdb389 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:34.025168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca89611-c3b8-4ddd-894a-ec77f1cdb389 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:34.027242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 556/1111] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:34.032529 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afa9b3-fb87-4256-bb9a-0f4bd3e59028 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:34.032968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afa9b3-fb87-4256-bb9a-0f4bd3e59028 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:34.033070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afa9b3-fb87-4256-bb9a-0f4bd3e59028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:34.050332 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f0afa9b3-fb87-4256-bb9a-0f4bd3e59028 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:34.051559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f0afa9b3-fb87-4256-bb9a-0f4bd3e59028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:34.117499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 556/1112] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:34.141876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca89611-c3b8-4ddd-894a-ec77f1cdb389 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:34.178083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:34.247099 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca89611-c3b8-4ddd-894a-ec77f1cdb389 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:34.288698 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd316e-726b-4642-bdfe-e6ff6ba96e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:34.289829 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd316e-726b-4642-bdfe-e6ff6ba96e32 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:34.290173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd316e-726b-4642-bdfe-e6ff6ba96e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:34.290688 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd316e-726b-4642-bdfe-e6ff6ba96e32 None admin] PATH_INFO: `/v3/roles/5addc8ffe20f47749792ac1394f6235c` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:34.292807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd316e-726b-4642-bdfe-e6ff6ba96e32 None admin] RBAC: Authorizing `identity:get_role(role_id=5addc8ffe20f47749792ac1394f6235c)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:34.294369 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd316e-726b-4642-bdfe-e6ff6ba96e32 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:34.295904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 557/1113] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:23:34 2022] GET /identity/v3/roles/5addc8ffe20f47749792ac1394f6235c => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:34.301762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afa9b3-fb87-4256-bb9a-0f4bd3e59028 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:34.361813 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:34.404333 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afa9b3-fb87-4256-bb9a-0f4bd3e59028 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:34.439096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726d0fb-f33c-4216-8734-eccf0b0d04ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:34.439884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726d0fb-f33c-4216-8734-eccf0b0d04ae None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:34.440167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726d0fb-f33c-4216-8734-eccf0b0d04ae None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:34.440447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726d0fb-f33c-4216-8734-eccf0b0d04ae None admin] PATH_INFO: `/v3/users/1a88211d748a414a9c00e0597001c453` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:34.454618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726d0fb-f33c-4216-8734-eccf0b0d04ae None admin] RBAC: Authorizing `identity:get_user(user_id=1a88211d748a414a9c00e0597001c453)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:34.456066 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726d0fb-f33c-4216-8734-eccf0b0d04ae None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:34.457826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 557/1114] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:23:34 2022] GET /identity/v3/users/1a88211d748a414a9c00e0597001c453 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:34.464431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd316e-726b-4642-bdfe-e6ff6ba96e32 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:34.571774 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd316e-726b-4642-bdfe-e6ff6ba96e32 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:34.573556 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:34.608106 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a106dbc-df59-4ea8-8ed4-bc9d8a5b1b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:34.608757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a106dbc-df59-4ea8-8ed4-bc9d8a5b1b39 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:34.609049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a106dbc-df59-4ea8-8ed4-bc9d8a5b1b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:34.609354 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a106dbc-df59-4ea8-8ed4-bc9d8a5b1b39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:34.677467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a106dbc-df59-4ea8-8ed4-bc9d8a5b1b39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:34.678930 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a106dbc-df59-4ea8-8ed4-bc9d8a5b1b39 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:34.776409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 558/1115] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:34.782357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726d0fb-f33c-4216-8734-eccf0b0d04ae None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:34.879120 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726d0fb-f33c-4216-8734-eccf0b0d04ae None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:34.897642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:34.915028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57dfae-f8d6-4600-a096-92956c4e6316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:34.916568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57dfae-f8d6-4600-a096-92956c4e6316 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:34.916874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57dfae-f8d6-4600-a096-92956c4e6316 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:34.917190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57dfae-f8d6-4600-a096-92956c4e6316 None admin] PATH_INFO: `/v3/projects/96b655f8d0934bada5d6354560d488f2` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:34.918433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57dfae-f8d6-4600-a096-92956c4e6316 None admin] RBAC: Authorizing `identity:get_project(project_id=96b655f8d0934bada5d6354560d488f2)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:34.919866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57dfae-f8d6-4600-a096-92956c4e6316 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:34.921628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 558/1116] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:23:34 2022] GET /identity/v3/projects/96b655f8d0934bada5d6354560d488f2 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:34.927812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a106dbc-df59-4ea8-8ed4-bc9d8a5b1b39 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:34.930038 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:35.031395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a106dbc-df59-4ea8-8ed4-bc9d8a5b1b39 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:35.065625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962df8a-d201-4595-9b6b-2f4079301040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:35.066278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962df8a-d201-4595-9b6b-2f4079301040 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:35.066563 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962df8a-d201-4595-9b6b-2f4079301040 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:35.066838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962df8a-d201-4595-9b6b-2f4079301040 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:35.067666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962df8a-d201-4595-9b6b-2f4079301040 None admin] RBAC: Adding query filter params (role.id=5addc8ffe20f47749792ac1394f6235c, scope.project.id=96b655f8d0934bada5d6354560d488f2, user.id=1a88211d748a414a9c00e0597001c453) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:35.067980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962df8a-d201-4595-9b6b-2f4079301040 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:35.069429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962df8a-d201-4595-9b6b-2f4079301040 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:35.075231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 559/1117] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:23:34 2022] GET /identity/v3/role_assignments?user.id=1a88211d748a414a9c00e0597001c453&scope.project.id=96b655f8d0934bada5d6354560d488f2&role.id=5addc8ffe20f47749792ac1394f6235c => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:37.339835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3895311-c696-495f-9423-b3feccd8b9b9 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:37.340363 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3895311-c696-495f-9423-b3feccd8b9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:37.340363 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3895311-c696-495f-9423-b3feccd8b9b9 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:37.341680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 559/1118] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:37.347017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc65db7-7edf-4fcb-8dd5-1c14f555e898 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:37.347464 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc65db7-7edf-4fcb-8dd5-1c14f555e898 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:37.347573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc65db7-7edf-4fcb-8dd5-1c14f555e898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:37.364436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-afc65db7-7edf-4fcb-8dd5-1c14f555e898 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:37.365717 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-afc65db7-7edf-4fcb-8dd5-1c14f555e898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:37.425030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:37.434116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 560/1119] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:37.457628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3895311-c696-495f-9423-b3feccd8b9b9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:37.558566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3895311-c696-495f-9423-b3feccd8b9b9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:37.593387 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0230dc-5c28-4759-81ff-e072db1bb36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:37.594156 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0230dc-5c28-4759-81ff-e072db1bb36b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:37.594435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0230dc-5c28-4759-81ff-e072db1bb36b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:37.594705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0230dc-5c28-4759-81ff-e072db1bb36b None admin] PATH_INFO: `/v3/roles/5addc8ffe20f47749792ac1394f6235c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:37.596155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0230dc-5c28-4759-81ff-e072db1bb36b None admin] RBAC: Authorizing `identity:get_role(role_id=5addc8ffe20f47749792ac1394f6235c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:37.597545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0230dc-5c28-4759-81ff-e072db1bb36b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:37.598899 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 560/1120] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:23:37 2022] GET /identity/v3/roles/5addc8ffe20f47749792ac1394f6235c => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:37.605404 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc65db7-7edf-4fcb-8dd5-1c14f555e898 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:37.704045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc65db7-7edf-4fcb-8dd5-1c14f555e898 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:37.737670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b45ede-d942-493e-8610-36c9de40fde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:37.738556 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b45ede-d942-493e-8610-36c9de40fde6 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:37.738838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b45ede-d942-493e-8610-36c9de40fde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:37.739113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b45ede-d942-493e-8610-36c9de40fde6 None admin] PATH_INFO: `/v3/users/1a88211d748a414a9c00e0597001c453` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:37.740307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b45ede-d942-493e-8610-36c9de40fde6 None admin] RBAC: Authorizing `identity:get_user(user_id=1a88211d748a414a9c00e0597001c453)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:37.741799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b45ede-d942-493e-8610-36c9de40fde6 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:37.744217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 561/1121] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:23:37 2022] GET /identity/v3/users/1a88211d748a414a9c00e0597001c453 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:37.750010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0230dc-5c28-4759-81ff-e072db1bb36b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:37.804261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:37.848961 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0230dc-5c28-4759-81ff-e072db1bb36b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:37.884418 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002977b7-cb91-408a-a4ef-bfaf6713da8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:37.885299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002977b7-cb91-408a-a4ef-bfaf6713da8f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:37.885580 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002977b7-cb91-408a-a4ef-bfaf6713da8f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:37.885858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002977b7-cb91-408a-a4ef-bfaf6713da8f None admin] PATH_INFO: `/v3/projects/96b655f8d0934bada5d6354560d488f2` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:37.887106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002977b7-cb91-408a-a4ef-bfaf6713da8f None admin] RBAC: Authorizing `identity:get_project(project_id=96b655f8d0934bada5d6354560d488f2)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:37.888544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002977b7-cb91-408a-a4ef-bfaf6713da8f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:37.890331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 561/1122] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:23:37 2022] GET /identity/v3/projects/96b655f8d0934bada5d6354560d488f2 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:37.896960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b45ede-d942-493e-8610-36c9de40fde6 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:37.967897 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:38.006033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b45ede-d942-493e-8610-36c9de40fde6 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:38.039940 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae56f50-38ff-4911-bfd6-c4d7856d2bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:38.041408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae56f50-38ff-4911-bfd6-c4d7856d2bea None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:38.041697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae56f50-38ff-4911-bfd6-c4d7856d2bea None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:38.041970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae56f50-38ff-4911-bfd6-c4d7856d2bea None admin] PATH_INFO: `/v3/projects/96b655f8d0934bada5d6354560d488f2/users/1a88211d748a414a9c00e0597001c453/roles/5addc8ffe20f47749792ac1394f6235c` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:38.044171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae56f50-38ff-4911-bfd6-c4d7856d2bea None admin] RBAC: Authorizing `identity:create_grant(project_id=96b655f8d0934bada5d6354560d488f2, user_id=1a88211d748a414a9c00e0597001c453, role_id=5addc8ffe20f47749792ac1394f6235c)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:38.045642 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae56f50-38ff-4911-bfd6-c4d7856d2bea None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:38.058776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 562/1123] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:23:37 2022] PUT /identity/v3/projects/96b655f8d0934bada5d6354560d488f2/users/1a88211d748a414a9c00e0597001c453/roles/5addc8ffe20f47749792ac1394f6235c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:23:40.312348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848933d8-13eb-4943-bd2c-2f1090ce27a1 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:40.312868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848933d8-13eb-4943-bd2c-2f1090ce27a1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:40.312868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848933d8-13eb-4943-bd2c-2f1090ce27a1 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:40.314186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 562/1124] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:40.319379 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0478497-739c-4e58-94f5-e0b6fc288ca2 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:40.319815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0478497-739c-4e58-94f5-e0b6fc288ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:40.319953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0478497-739c-4e58-94f5-e0b6fc288ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:40.336728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-f0478497-739c-4e58-94f5-e0b6fc288ca2 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:40.338650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f0478497-739c-4e58-94f5-e0b6fc288ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:40.404330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 563/1125] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:40.428313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848933d8-13eb-4943-bd2c-2f1090ce27a1 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:40.528129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848933d8-13eb-4943-bd2c-2f1090ce27a1 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:40.551520 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:40.563983 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bba08-df0b-4cdc-ac41-dcb58f755055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:40.564639 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bba08-df0b-4cdc-ac41-dcb58f755055 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:40.564924 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bba08-df0b-4cdc-ac41-dcb58f755055 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:40.565263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bba08-df0b-4cdc-ac41-dcb58f755055 None admin] PATH_INFO: `/v3/roles/5addc8ffe20f47749792ac1394f6235c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:40.566740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bba08-df0b-4cdc-ac41-dcb58f755055 None admin] RBAC: Authorizing `identity:get_role(role_id=5addc8ffe20f47749792ac1394f6235c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:40.568089 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bba08-df0b-4cdc-ac41-dcb58f755055 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:40.569491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 563/1126] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:23:40 2022] GET /identity/v3/roles/5addc8ffe20f47749792ac1394f6235c => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:40.575431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0478497-739c-4e58-94f5-e0b6fc288ca2 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:40.630574 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:40.676153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0478497-739c-4e58-94f5-e0b6fc288ca2 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:40.710081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7ec2f-c7f7-43bc-853d-b674ca2bede5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:40.710941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7ec2f-c7f7-43bc-853d-b674ca2bede5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:40.711221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7ec2f-c7f7-43bc-853d-b674ca2bede5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:40.711491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7ec2f-c7f7-43bc-853d-b674ca2bede5 None admin] PATH_INFO: `/v3/users/1a88211d748a414a9c00e0597001c453` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:40.713321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7ec2f-c7f7-43bc-853d-b674ca2bede5 None admin] RBAC: Authorizing `identity:get_user(user_id=1a88211d748a414a9c00e0597001c453)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:40.714745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7ec2f-c7f7-43bc-853d-b674ca2bede5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:40.716386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 564/1127] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:23:40 2022] GET /identity/v3/users/1a88211d748a414a9c00e0597001c453 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:40.722131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bba08-df0b-4cdc-ac41-dcb58f755055 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:40.818255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bba08-df0b-4cdc-ac41-dcb58f755055 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:40.851770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71041d-bc5c-4526-b757-2d624ca2bf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:40.852404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71041d-bc5c-4526-b757-2d624ca2bf7f None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:40.853253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71041d-bc5c-4526-b757-2d624ca2bf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:40.853498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71041d-bc5c-4526-b757-2d624ca2bf7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:40.912241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:40.923582 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71041d-bc5c-4526-b757-2d624ca2bf7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:40.925047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71041d-bc5c-4526-b757-2d624ca2bf7f None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:41.028502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 564/1128] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:41.034445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7ec2f-c7f7-43bc-853d-b674ca2bede5 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:41.134240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7ec2f-c7f7-43bc-853d-b674ca2bede5 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:41.157660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:41.170579 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9724bf-1943-41d2-8cc4-3ffc60a9729a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:41.171446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9724bf-1943-41d2-8cc4-3ffc60a9729a None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:41.171733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9724bf-1943-41d2-8cc4-3ffc60a9729a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:41.172014 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9724bf-1943-41d2-8cc4-3ffc60a9729a None admin] PATH_INFO: `/v3/projects/96b655f8d0934bada5d6354560d488f2` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:41.173240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9724bf-1943-41d2-8cc4-3ffc60a9729a None admin] RBAC: Authorizing `identity:get_project(project_id=96b655f8d0934bada5d6354560d488f2)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:41.174689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9724bf-1943-41d2-8cc4-3ffc60a9729a None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:41.176427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 565/1129] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:23:41 2022] GET /identity/v3/projects/96b655f8d0934bada5d6354560d488f2 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:41.183372 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71041d-bc5c-4526-b757-2d624ca2bf7f None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:41.281482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:41.292093 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71041d-bc5c-4526-b757-2d624ca2bf7f None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:41.325402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e09698-5127-4327-a010-168c76d3dccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:41.326070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e09698-5127-4327-a010-168c76d3dccf None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:41.326314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e09698-5127-4327-a010-168c76d3dccf None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:41.326586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e09698-5127-4327-a010-168c76d3dccf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:41.327405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e09698-5127-4327-a010-168c76d3dccf None admin] RBAC: Adding query filter params (role.id=5addc8ffe20f47749792ac1394f6235c, scope.project.id=96b655f8d0934bada5d6354560d488f2, user.id=1a88211d748a414a9c00e0597001c453) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:41.327714 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e09698-5127-4327-a010-168c76d3dccf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:41.329416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e09698-5127-4327-a010-168c76d3dccf None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:41.335299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 565/1130] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:23:41 2022] GET /identity/v3/role_assignments?user.id=1a88211d748a414a9c00e0597001c453&scope.project.id=96b655f8d0934bada5d6354560d488f2&role.id=5addc8ffe20f47749792ac1394f6235c => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:43.673665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3814e07-7881-4ea0-9454-fc28526df181 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:43.674196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3814e07-7881-4ea0-9454-fc28526df181 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:43.674292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3814e07-7881-4ea0-9454-fc28526df181 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:43.675596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 566/1131] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:43.681261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c770de7-ae93-4846-b97c-14bdc1693bcd None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:43.681654 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c770de7-ae93-4846-b97c-14bdc1693bcd None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:43.681917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c770de7-ae93-4846-b97c-14bdc1693bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:43.698988 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0c770de7-ae93-4846-b97c-14bdc1693bcd None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:43.700175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0c770de7-ae93-4846-b97c-14bdc1693bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:43.766950 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 566/1132] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:43.791550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3814e07-7881-4ea0-9454-fc28526df181 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:43.899092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3814e07-7881-4ea0-9454-fc28526df181 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:43.934388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9934a7-3ff5-40f4-aa4d-adaee380ddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:43.935091 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9934a7-3ff5-40f4-aa4d-adaee380ddcd None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:43.935375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9934a7-3ff5-40f4-aa4d-adaee380ddcd None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:43.935651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9934a7-3ff5-40f4-aa4d-adaee380ddcd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:43.936819 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9934a7-3ff5-40f4-aa4d-adaee380ddcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:43.938292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9934a7-3ff5-40f4-aa4d-adaee380ddcd None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:43.939914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 567/1133] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:43.947007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c770de7-ae93-4846-b97c-14bdc1693bcd None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:44.030229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:44.046615 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c770de7-ae93-4846-b97c-14bdc1693bcd None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:44.080288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f2788-20ff-4813-a7b4-11137c87e5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:44.080972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f2788-20ff-4813-a7b4-11137c87e5bc None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:44.081288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f2788-20ff-4813-a7b4-11137c87e5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:44.081570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f2788-20ff-4813-a7b4-11137c87e5bc None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:44.082305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f2788-20ff-4813-a7b4-11137c87e5bc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:44.083772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f2788-20ff-4813-a7b4-11137c87e5bc None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:44.111896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 567/1134] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:23:43 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:23:46.426737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbdfb1-1fee-4ced-82a9-9a105baf0251 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:46.427298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbdfb1-1fee-4ced-82a9-9a105baf0251 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:46.427298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbdfb1-1fee-4ced-82a9-9a105baf0251 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:46.428655 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 568/1135] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:46.434882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66963db2-98ea-4634-9c0e-464aa344bf2b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:46.435283 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66963db2-98ea-4634-9c0e-464aa344bf2b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:46.435446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66963db2-98ea-4634-9c0e-464aa344bf2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:46.454084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-66963db2-98ea-4634-9c0e-464aa344bf2b None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:46.455292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-66963db2-98ea-4634-9c0e-464aa344bf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:46.527320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 568/1136] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:46.552409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbdfb1-1fee-4ced-82a9-9a105baf0251 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:46.613214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:46.659870 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbdfb1-1fee-4ced-82a9-9a105baf0251 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:46.696485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895cdf1-9f7e-460d-b524-7eb5edc6d8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:46.697208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895cdf1-9f7e-460d-b524-7eb5edc6d8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:46.697496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895cdf1-9f7e-460d-b524-7eb5edc6d8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:46.697772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895cdf1-9f7e-460d-b524-7eb5edc6d8a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:46.698953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895cdf1-9f7e-460d-b524-7eb5edc6d8a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:46.700393 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895cdf1-9f7e-460d-b524-7eb5edc6d8a6 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:46.702172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 569/1137] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:46.708217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66963db2-98ea-4634-9c0e-464aa344bf2b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:46.772997 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:46.808833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66963db2-98ea-4634-9c0e-464aa344bf2b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:46.843033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ee910-f957-44f9-a3a5-8720fb545d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:46.843710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ee910-f957-44f9-a3a5-8720fb545d89 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:46.843991 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ee910-f957-44f9-a3a5-8720fb545d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:46.844268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ee910-f957-44f9-a3a5-8720fb545d89 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:46.844983 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ee910-f957-44f9-a3a5-8720fb545d89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:46.846457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ee910-f957-44f9-a3a5-8720fb545d89 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:46.866346 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 569/1138] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:23:46 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:23:49.191825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce9ede-f90e-4e4b-af96-f5d3dcd7fb8f None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:49.192348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce9ede-f90e-4e4b-af96-f5d3dcd7fb8f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:49.192348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce9ede-f90e-4e4b-af96-f5d3dcd7fb8f None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:49.193679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 570/1139] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:49.199515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732b151-330a-448e-a45c-f4bde54da81e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:49.199940 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732b151-330a-448e-a45c-f4bde54da81e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:49.200078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732b151-330a-448e-a45c-f4bde54da81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:49.217648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-8732b151-330a-448e-a45c-f4bde54da81e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:49.218846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8732b151-330a-448e-a45c-f4bde54da81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:49.286273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 570/1140] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:49.310910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce9ede-f90e-4e4b-af96-f5d3dcd7fb8f None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:49.412630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce9ede-f90e-4e4b-af96-f5d3dcd7fb8f None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:49.444295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:49.455572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335e628-a1de-4c62-b93e-c86fe6706c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:49.456356 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335e628-a1de-4c62-b93e-c86fe6706c89 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:49.456636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335e628-a1de-4c62-b93e-c86fe6706c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:49.456911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335e628-a1de-4c62-b93e-c86fe6706c89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:49.468129 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335e628-a1de-4c62-b93e-c86fe6706c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-e335e628-a1de-4c62-b93e-c86fe6706c89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:49.475605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:49.476773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:49.477931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:49.477931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:49.477931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:49.477931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:49.477931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:49.477931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:49.477931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:49.477931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:49.477931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:49.477931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:49.477931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:49.477931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:49.477931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 571/1141] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:23:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:49.483557 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732b151-330a-448e-a45c-f4bde54da81e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:49.534547 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:49.582070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732b151-330a-448e-a45c-f4bde54da81e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:49.616391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54589c9f-0c08-4812-9fc8-6d662fbf88ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:49.617057 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54589c9f-0c08-4812-9fc8-6d662fbf88ee None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:49.617373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54589c9f-0c08-4812-9fc8-6d662fbf88ee None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:49.617651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54589c9f-0c08-4812-9fc8-6d662fbf88ee None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:49.618569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54589c9f-0c08-4812-9fc8-6d662fbf88ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:49.618881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54589c9f-0c08-4812-9fc8-6d662fbf88ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:49.620239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54589c9f-0c08-4812-9fc8-6d662fbf88ee None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:49.628342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 571/1142] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:23:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:49.634838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335e628-a1de-4c62-b93e-c86fe6706c89 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:49.734097 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335e628-a1de-4c62-b93e-c86fe6706c89 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:49.770235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78098a-647d-42c1-8b18-1518b9ef558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:49.771109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78098a-647d-42c1-8b18-1518b9ef558c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:49.771394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78098a-647d-42c1-8b18-1518b9ef558c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:49.771668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78098a-647d-42c1-8b18-1518b9ef558c None admin] PATH_INFO: `/v3/users/906f60606c534fee845aa0865b46ec05` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:49.785439 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78098a-647d-42c1-8b18-1518b9ef558c None admin] RBAC: Authorizing `identity:get_user(user_id=906f60606c534fee845aa0865b46ec05)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:49.786901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78098a-647d-42c1-8b18-1518b9ef558c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:49.788642 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 572/1143] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:23:49 2022] GET /identity/v3/users/906f60606c534fee845aa0865b46ec05 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:49.795554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54589c9f-0c08-4812-9fc8-6d662fbf88ee None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:49.900938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54589c9f-0c08-4812-9fc8-6d662fbf88ee None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:49.914625 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:49.935671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38459628-2153-4b0c-a130-09b78d421d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:49.936314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38459628-2153-4b0c-a130-09b78d421d43 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:49.936592 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38459628-2153-4b0c-a130-09b78d421d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:49.936862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38459628-2153-4b0c-a130-09b78d421d43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:50.003960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38459628-2153-4b0c-a130-09b78d421d43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:50.005357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38459628-2153-4b0c-a130-09b78d421d43 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:50.116268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 572/1144] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:50.122890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78098a-647d-42c1-8b18-1518b9ef558c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:50.174614 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:50.233350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78098a-647d-42c1-8b18-1518b9ef558c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:50.269892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0dfe2-be4a-400f-81fe-31f0b3c11ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:50.270664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0dfe2-be4a-400f-81fe-31f0b3c11ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:50.270947 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0dfe2-be4a-400f-81fe-31f0b3c11ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:50.271220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0dfe2-be4a-400f-81fe-31f0b3c11ea0 None admin] PATH_INFO: `/v3/projects/2df89e3158c44f62bceaf9eb347e1a70` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:50.272387 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0dfe2-be4a-400f-81fe-31f0b3c11ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=2df89e3158c44f62bceaf9eb347e1a70)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:50.273843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0dfe2-be4a-400f-81fe-31f0b3c11ea0 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:50.276505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 573/1145] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:23:50 2022] GET /identity/v3/projects/2df89e3158c44f62bceaf9eb347e1a70 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:50.282599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38459628-2153-4b0c-a130-09b78d421d43 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:50.298291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:50.390761 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38459628-2153-4b0c-a130-09b78d421d43 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:50.425832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d96f3-5cd9-43d1-acc2-56c1b83e8d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:50.426485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d96f3-5cd9-43d1-acc2-56c1b83e8d81 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:50.426768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d96f3-5cd9-43d1-acc2-56c1b83e8d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:50.427044 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d96f3-5cd9-43d1-acc2-56c1b83e8d81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:50.427865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d96f3-5cd9-43d1-acc2-56c1b83e8d81 None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=2df89e3158c44f62bceaf9eb347e1a70, user.id=906f60606c534fee845aa0865b46ec05) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:50.428176 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d96f3-5cd9-43d1-acc2-56c1b83e8d81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:50.429634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d96f3-5cd9-43d1-acc2-56c1b83e8d81 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:50.435384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 573/1146] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:23:50 2022] GET /identity/v3/role_assignments?user.id=906f60606c534fee845aa0865b46ec05&scope.project.id=2df89e3158c44f62bceaf9eb347e1a70&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:52.702034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6ead8-2e72-4890-aa8e-7ca473d43478 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:52.702600 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6ead8-2e72-4890-aa8e-7ca473d43478 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:52.702600 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6ead8-2e72-4890-aa8e-7ca473d43478 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:52.704269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 574/1147] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:23:52.709689 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ece5df-633d-4ff5-ba26-4c9162091f18 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:52.710125 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ece5df-633d-4ff5-ba26-4c9162091f18 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:52.710219 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ece5df-633d-4ff5-ba26-4c9162091f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:52.726778 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-21ece5df-633d-4ff5-ba26-4c9162091f18 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:52.728500 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-21ece5df-633d-4ff5-ba26-4c9162091f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:52.794338 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 574/1148] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:52.817378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6ead8-2e72-4890-aa8e-7ca473d43478 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:52.926265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6ead8-2e72-4890-aa8e-7ca473d43478 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:52.940156 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:52.961315 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1146a-06be-473c-a585-2361f6adc1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:52.962565 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1146a-06be-473c-a585-2361f6adc1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:52.962848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1146a-06be-473c-a585-2361f6adc1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:52.963124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1146a-06be-473c-a585-2361f6adc1a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:52.973336 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d1146a-06be-473c-a585-2361f6adc1a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-79d1146a-06be-473c-a585-2361f6adc1a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:52.981519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:52.982657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:52.983644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:52.983644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:52.983644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:52.983644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:52.983644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:52.983644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:52.983644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:52.983644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:52.983644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:52.983644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:52.983644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:52.983644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:23:52.983644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 575/1149] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:23:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:52.988911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ece5df-633d-4ff5-ba26-4c9162091f18 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:52.999649 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:53.099566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ece5df-633d-4ff5-ba26-4c9162091f18 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:53.132731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb266b-c612-4a7c-b9f7-eb6054177b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:53.133491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb266b-c612-4a7c-b9f7-eb6054177b50 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:53.133931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb266b-c612-4a7c-b9f7-eb6054177b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:53.134229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb266b-c612-4a7c-b9f7-eb6054177b50 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:53.134982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb266b-c612-4a7c-b9f7-eb6054177b50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:53.135289 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb266b-c612-4a7c-b9f7-eb6054177b50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:53.136648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb266b-c612-4a7c-b9f7-eb6054177b50 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:53.144882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 575/1150] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:23:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:53.151520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1146a-06be-473c-a585-2361f6adc1a8 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:53.251617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1146a-06be-473c-a585-2361f6adc1a8 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:53.284617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f3b62-0731-4066-9002-2ec51dbac885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:53.285503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f3b62-0731-4066-9002-2ec51dbac885 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:53.285789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f3b62-0731-4066-9002-2ec51dbac885 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:53.286065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f3b62-0731-4066-9002-2ec51dbac885 None admin] PATH_INFO: `/v3/users/906f60606c534fee845aa0865b46ec05` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:53.287249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f3b62-0731-4066-9002-2ec51dbac885 None admin] RBAC: Authorizing `identity:get_user(user_id=906f60606c534fee845aa0865b46ec05)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:53.288698 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f3b62-0731-4066-9002-2ec51dbac885 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:53.290490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 576/1151] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:23:53 2022] GET /identity/v3/users/906f60606c534fee845aa0865b46ec05 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:53.296320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb266b-c612-4a7c-b9f7-eb6054177b50 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:53.365490 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:53.396844 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb266b-c612-4a7c-b9f7-eb6054177b50 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:53.429849 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14481e69-d738-4c33-a579-632801dbd224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:53.430915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14481e69-d738-4c33-a579-632801dbd224 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:53.431201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14481e69-d738-4c33-a579-632801dbd224 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:53.431486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14481e69-d738-4c33-a579-632801dbd224 None admin] PATH_INFO: `/v3/projects/2df89e3158c44f62bceaf9eb347e1a70` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:53.432671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14481e69-d738-4c33-a579-632801dbd224 None admin] RBAC: Authorizing `identity:get_project(project_id=2df89e3158c44f62bceaf9eb347e1a70)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:53.434123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14481e69-d738-4c33-a579-632801dbd224 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:53.435836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 576/1152] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:23:53 2022] GET /identity/v3/projects/2df89e3158c44f62bceaf9eb347e1a70 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:53.441780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f3b62-0731-4066-9002-2ec51dbac885 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:53.485006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:53.543294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f3b62-0731-4066-9002-2ec51dbac885 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:53.577174 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c2bfb-308c-4422-b668-c809a724a1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:53.578065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c2bfb-308c-4422-b668-c809a724a1d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:53.578346 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c2bfb-308c-4422-b668-c809a724a1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:53.578880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c2bfb-308c-4422-b668-c809a724a1d9 None admin] PATH_INFO: `/v3/projects/2df89e3158c44f62bceaf9eb347e1a70/users/906f60606c534fee845aa0865b46ec05/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:53.580833 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c2bfb-308c-4422-b668-c809a724a1d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2df89e3158c44f62bceaf9eb347e1a70, user_id=906f60606c534fee845aa0865b46ec05, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:53.582315 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c2bfb-308c-4422-b668-c809a724a1d9 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:53.596429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 577/1153] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:23:53 2022] PUT /identity/v3/projects/2df89e3158c44f62bceaf9eb347e1a70/users/906f60606c534fee845aa0865b46ec05/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:23:55.860319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ac0e6-3e68-43a1-b105-ba7065065536 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:55.860854 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ac0e6-3e68-43a1-b105-ba7065065536 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:55.860854 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ac0e6-3e68-43a1-b105-ba7065065536 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:55.862399 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 577/1154] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:55.868325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4622869-c46c-4956-972c-74b6e85c05ab None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:55.868759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4622869-c46c-4956-972c-74b6e85c05ab None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:55.868903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4622869-c46c-4956-972c-74b6e85c05ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:55.885582 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-f4622869-c46c-4956-972c-74b6e85c05ab None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:55.886794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f4622869-c46c-4956-972c-74b6e85c05ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:55.959197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 578/1155] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:55.982905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ac0e6-3e68-43a1-b105-ba7065065536 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:56.086338 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ac0e6-3e68-43a1-b105-ba7065065536 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:56.120263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2979cf-8ee5-4181-a1c6-0b533696fdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:56.121068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2979cf-8ee5-4181-a1c6-0b533696fdef None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:56.121555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2979cf-8ee5-4181-a1c6-0b533696fdef None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:56.121833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2979cf-8ee5-4181-a1c6-0b533696fdef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:56.128794 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:56.132705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2979cf-8ee5-4181-a1c6-0b533696fdef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-9c2979cf-8ee5-4181-a1c6-0b533696fdef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:56.141276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:56.142392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:56.143321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:56.143321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:56.143321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:56.143321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:56.143321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:56.143321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:56.143321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:56.143321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:56.143321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:56.143321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:56.143321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:56.143321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:23:56.143321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 578/1156] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:23:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:56.148600 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4622869-c46c-4956-972c-74b6e85c05ab None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:56.177476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:56.250560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4622869-c46c-4956-972c-74b6e85c05ab None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:56.284107 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16cf27-d2d7-4b10-bb72-b8b7c2af6db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:56.284793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16cf27-d2d7-4b10-bb72-b8b7c2af6db8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:56.285119 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16cf27-d2d7-4b10-bb72-b8b7c2af6db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:56.285363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16cf27-d2d7-4b10-bb72-b8b7c2af6db8 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:56.286114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16cf27-d2d7-4b10-bb72-b8b7c2af6db8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:56.286468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16cf27-d2d7-4b10-bb72-b8b7c2af6db8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:56.287829 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16cf27-d2d7-4b10-bb72-b8b7c2af6db8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:56.296264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 579/1157] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:23:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:56.302079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2979cf-8ee5-4181-a1c6-0b533696fdef None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:56.403923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2979cf-8ee5-4181-a1c6-0b533696fdef None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:56.436051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc3a67-8f8c-4f17-9fca-37c9e57295b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:56.436912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc3a67-8f8c-4f17-9fca-37c9e57295b2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:56.437223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc3a67-8f8c-4f17-9fca-37c9e57295b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:56.437503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc3a67-8f8c-4f17-9fca-37c9e57295b2 None admin] PATH_INFO: `/v3/users/906f60606c534fee845aa0865b46ec05` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:56.438683 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc3a67-8f8c-4f17-9fca-37c9e57295b2 None admin] RBAC: Authorizing `identity:get_user(user_id=906f60606c534fee845aa0865b46ec05)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:56.440097 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc3a67-8f8c-4f17-9fca-37c9e57295b2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:56.441744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 579/1158] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:23:56 2022] GET /identity/v3/users/906f60606c534fee845aa0865b46ec05 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:56.448158 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16cf27-d2d7-4b10-bb72-b8b7c2af6db8 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:56.547162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:56.556329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16cf27-d2d7-4b10-bb72-b8b7c2af6db8 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:56.591323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05606e69-b549-46bb-bc6f-ea5a4309885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:56.591964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05606e69-b549-46bb-bc6f-ea5a4309885b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:56.592249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05606e69-b549-46bb-bc6f-ea5a4309885b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:56.592525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05606e69-b549-46bb-bc6f-ea5a4309885b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:56.658993 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05606e69-b549-46bb-bc6f-ea5a4309885b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:56.660360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05606e69-b549-46bb-bc6f-ea5a4309885b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:56.749413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:56.759617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 580/1159] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:56.765582 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc3a67-8f8c-4f17-9fca-37c9e57295b2 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:56.830684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:56.870664 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc3a67-8f8c-4f17-9fca-37c9e57295b2 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:56.916145 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a0d57-df07-48c2-a5b1-07478b6a6f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:56.916954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a0d57-df07-48c2-a5b1-07478b6a6f91 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:56.917274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a0d57-df07-48c2-a5b1-07478b6a6f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:56.917552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a0d57-df07-48c2-a5b1-07478b6a6f91 None admin] PATH_INFO: `/v3/projects/2df89e3158c44f62bceaf9eb347e1a70` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:56.918820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a0d57-df07-48c2-a5b1-07478b6a6f91 None admin] RBAC: Authorizing `identity:get_project(project_id=2df89e3158c44f62bceaf9eb347e1a70)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:56.920257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a0d57-df07-48c2-a5b1-07478b6a6f91 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:56.922038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 580/1160] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:23:56 2022] GET /identity/v3/projects/2df89e3158c44f62bceaf9eb347e1a70 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:56.928277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05606e69-b549-46bb-bc6f-ea5a4309885b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:57.026996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05606e69-b549-46bb-bc6f-ea5a4309885b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:57.060476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e149a31-5ab9-42a5-aa0a-02d2d3fe454b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:57.061144 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e149a31-5ab9-42a5-aa0a-02d2d3fe454b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:57.061426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e149a31-5ab9-42a5-aa0a-02d2d3fe454b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:57.061697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e149a31-5ab9-42a5-aa0a-02d2d3fe454b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:57.062512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e149a31-5ab9-42a5-aa0a-02d2d3fe454b None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=2df89e3158c44f62bceaf9eb347e1a70, user.id=906f60606c534fee845aa0865b46ec05) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:57.062819 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e149a31-5ab9-42a5-aa0a-02d2d3fe454b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:57.064243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e149a31-5ab9-42a5-aa0a-02d2d3fe454b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:57.070018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 581/1161] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:23:56 2022] GET /identity/v3/role_assignments?user.id=906f60606c534fee845aa0865b46ec05&scope.project.id=2df89e3158c44f62bceaf9eb347e1a70&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:59.427040 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff55ed-c9f9-4faf-8fd4-628c72494c97 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:59.427599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff55ed-c9f9-4faf-8fd4-628c72494c97 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:59.427599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff55ed-c9f9-4faf-8fd4-628c72494c97 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:59.429002 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 581/1162] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:59.435187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d0cc6-017d-4fdc-b5c6-74e9ad998568 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:59.435723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d0cc6-017d-4fdc-b5c6-74e9ad998568 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:59.435723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d0cc6-017d-4fdc-b5c6-74e9ad998568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:59.454353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d45d0cc6-017d-4fdc-b5c6-74e9ad998568 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:59.455564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d45d0cc6-017d-4fdc-b5c6-74e9ad998568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:59.506207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:59.525149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 582/1163] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:59.547602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff55ed-c9f9-4faf-8fd4-628c72494c97 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:59.652533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff55ed-c9f9-4faf-8fd4-628c72494c97 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:59.685482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd4f89-c6b9-4bc5-ab20-ab66b6d55731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:59.686054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd4f89-c6b9-4bc5-ab20-ab66b6d55731 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:59.686339 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd4f89-c6b9-4bc5-ab20-ab66b6d55731 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:59.686614 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd4f89-c6b9-4bc5-ab20-ab66b6d55731 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:59.689331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:59.755010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd4f89-c6b9-4bc5-ab20-ab66b6d55731 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:59.756463 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd4f89-c6b9-4bc5-ab20-ab66b6d55731 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:59.858217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 582/1164] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:59.864076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d0cc6-017d-4fdc-b5c6-74e9ad998568 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:59.968540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d0cc6-017d-4fdc-b5c6-74e9ad998568 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:00.009111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27434ef2-fc92-4d66-99b7-39897bcf83e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:00.010142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27434ef2-fc92-4d66-99b7-39897bcf83e5 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:00.010427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27434ef2-fc92-4d66-99b7-39897bcf83e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:00.010706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27434ef2-fc92-4d66-99b7-39897bcf83e5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:00.018446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27434ef2-fc92-4d66-99b7-39897bcf83e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:00.019926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27434ef2-fc92-4d66-99b7-39897bcf83e5 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-27434ef2-fc92-4d66-99b7-39897bcf83e5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:24:00.035786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:24:00.037073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:24:00.037989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 583/1165] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jun 7 08:23:59 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:24:00.044087 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd4f89-c6b9-4bc5-ab20-ab66b6d55731 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:00.080126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:00.151544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd4f89-c6b9-4bc5-ab20-ab66b6d55731 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:00.185782 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649acdc-41ba-4591-a18d-19e771258db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:00.186451 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649acdc-41ba-4591-a18d-19e771258db4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:00.186738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649acdc-41ba-4591-a18d-19e771258db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:00.187011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649acdc-41ba-4591-a18d-19e771258db4 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:00.187731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649acdc-41ba-4591-a18d-19e771258db4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:24:00.188034 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649acdc-41ba-4591-a18d-19e771258db4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:00.189419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649acdc-41ba-4591-a18d-19e771258db4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:00.197596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 583/1166] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jun 7 08:24:00 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:02.461737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22206f91-6083-4494-b568-dc6d00d95d5d None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:02.462382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22206f91-6083-4494-b568-dc6d00d95d5d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:02.462488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22206f91-6083-4494-b568-dc6d00d95d5d None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:02.463826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 584/1167] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:24:02.469736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bed6ff-194f-4a26-a232-2dfb15cabf51 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:02.470321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bed6ff-194f-4a26-a232-2dfb15cabf51 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:02.470321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bed6ff-194f-4a26-a232-2dfb15cabf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:02.489228 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-79bed6ff-194f-4a26-a232-2dfb15cabf51 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:24:02.490461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-79bed6ff-194f-4a26-a232-2dfb15cabf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:24:02.563385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 584/1168] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:24:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:24:02.587665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22206f91-6083-4494-b568-dc6d00d95d5d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:02.637528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:02.692269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22206f91-6083-4494-b568-dc6d00d95d5d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:02.726609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d08aa9-d2e7-4a7c-b950-04c96ed142a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:02.727805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d08aa9-d2e7-4a7c-b950-04c96ed142a8 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:02.728093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d08aa9-d2e7-4a7c-b950-04c96ed142a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:02.728362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d08aa9-d2e7-4a7c-b950-04c96ed142a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:02.729135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d08aa9-d2e7-4a7c-b950-04c96ed142a8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:02.730515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d08aa9-d2e7-4a7c-b950-04c96ed142a8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:02.745206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 585/1169] 10.222.0.39 () {60 vars in 1138 bytes} [Tue Jun 7 08:24:02 2022] POST /identity/v3/domains => generated 267 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:24:05.096752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90446f60-3911-4968-9f68-74284e2f3453 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:05.097358 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90446f60-3911-4968-9f68-74284e2f3453 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:05.097358 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90446f60-3911-4968-9f68-74284e2f3453 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:05.098720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 585/1170] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:24:05.104724 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10428ba1-8caa-4a5d-abf9-311aa6a4ebd9 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:05.105200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10428ba1-8caa-4a5d-abf9-311aa6a4ebd9 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:05.105309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10428ba1-8caa-4a5d-abf9-311aa6a4ebd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:05.121915 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-10428ba1-8caa-4a5d-abf9-311aa6a4ebd9 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:24:05.123114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-10428ba1-8caa-4a5d-abf9-311aa6a4ebd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:24:05.189945 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 586/1171] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:24:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:24:05.215122 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90446f60-3911-4968-9f68-74284e2f3453 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:05.247764 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:05.322311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90446f60-3911-4968-9f68-74284e2f3453 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:05.356689 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b09d5b-0a95-4955-84ba-74e6c8e5a543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:05.357566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b09d5b-0a95-4955-84ba-74e6c8e5a543 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:05.357851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b09d5b-0a95-4955-84ba-74e6c8e5a543 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:05.358125 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b09d5b-0a95-4955-84ba-74e6c8e5a543 None admin] PATH_INFO: `/v3/domains/c6464dccf5a74998b648d730a8b88f3c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:05.368541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b09d5b-0a95-4955-84ba-74e6c8e5a543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c6464dccf5a74998b648d730a8b88f3c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:05.370001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b09d5b-0a95-4955-84ba-74e6c8e5a543 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:05.371657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 586/1172] 10.222.0.39 () {56 vars in 1183 bytes} [Tue Jun 7 08:24:05 2022] GET /identity/v3/domains/c6464dccf5a74998b648d730a8b88f3c => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:05.377584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10428ba1-8caa-4a5d-abf9-311aa6a4ebd9 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:05.412479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:05.477630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10428ba1-8caa-4a5d-abf9-311aa6a4ebd9 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:05.520658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff4cbb-7f87-47c5-8320-b657f84419bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:05.521345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff4cbb-7f87-47c5-8320-b657f84419bb None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:05.521635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff4cbb-7f87-47c5-8320-b657f84419bb None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:05.521924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff4cbb-7f87-47c5-8320-b657f84419bb None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:05.522675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff4cbb-7f87-47c5-8320-b657f84419bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:05.524861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff4cbb-7f87-47c5-8320-b657f84419bb None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:05.551034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 587/1173] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:24:05 2022] POST /identity/v3/projects => generated 361 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:24:07.855074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e4f4d-59c1-47bb-8818-4e0ce40800ef None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:07.855644 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e4f4d-59c1-47bb-8818-4e0ce40800ef None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:07.855644 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e4f4d-59c1-47bb-8818-4e0ce40800ef None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:07.856999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 587/1174] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:24:07.862257 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c061c-f948-4f1b-a44b-f71b9f78265f None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:07.862688 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c061c-f948-4f1b-a44b-f71b9f78265f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:07.862823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c061c-f948-4f1b-a44b-f71b9f78265f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:07.879943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-b60c061c-f948-4f1b-a44b-f71b9f78265f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:24:07.881131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b60c061c-f948-4f1b-a44b-f71b9f78265f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:24:07.951914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 588/1175] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:24:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:24:07.977679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e4f4d-59c1-47bb-8818-4e0ce40800ef None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:08.068794 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:08.082571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e4f4d-59c1-47bb-8818-4e0ce40800ef None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:08.116946 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d5468-4e2c-4d68-8151-872541aa31e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:08.117864 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d5468-4e2c-4d68-8151-872541aa31e1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:08.118149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d5468-4e2c-4d68-8151-872541aa31e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:08.118426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d5468-4e2c-4d68-8151-872541aa31e1 None admin] PATH_INFO: `/v3/domains/c6464dccf5a74998b648d730a8b88f3c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:08.119597 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d5468-4e2c-4d68-8151-872541aa31e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c6464dccf5a74998b648d730a8b88f3c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:08.121013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d5468-4e2c-4d68-8151-872541aa31e1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:08.122662 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 588/1176] 10.222.0.39 () {56 vars in 1183 bytes} [Tue Jun 7 08:24:07 2022] GET /identity/v3/domains/c6464dccf5a74998b648d730a8b88f3c => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:08.128370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c061c-f948-4f1b-a44b-f71b9f78265f None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:08.146601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:08.227886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c061c-f948-4f1b-a44b-f71b9f78265f None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:08.260752 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758ede0-b4ae-4a74-9123-3fc930bd96e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:08.261447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758ede0-b4ae-4a74-9123-3fc930bd96e1 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:08.261726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758ede0-b4ae-4a74-9123-3fc930bd96e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:08.262001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758ede0-b4ae-4a74-9123-3fc930bd96e1 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:08.262713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758ede0-b4ae-4a74-9123-3fc930bd96e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:08.264136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758ede0-b4ae-4a74-9123-3fc930bd96e1 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:08.286457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 589/1177] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:24:08 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:24:10.578212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec6328-bfa1-425b-81e9-86f9bb8bdc10 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:10.579035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec6328-bfa1-425b-81e9-86f9bb8bdc10 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:10.579035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec6328-bfa1-425b-81e9-86f9bb8bdc10 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:10.580364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 589/1178] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:24:10.585698 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d95cd-f45d-48f8-9770-cd261c5bba4f None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:10.586195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d95cd-f45d-48f8-9770-cd261c5bba4f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:10.586459 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d95cd-f45d-48f8-9770-cd261c5bba4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:10.604348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-0a5d95cd-f45d-48f8-9770-cd261c5bba4f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:24:10.605569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0a5d95cd-f45d-48f8-9770-cd261c5bba4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:24:10.671428 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 590/1179] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:24:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:24:10.695531 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec6328-bfa1-425b-81e9-86f9bb8bdc10 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:10.798850 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec6328-bfa1-425b-81e9-86f9bb8bdc10 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:10.834552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a91e9c-10e7-448f-bd90-1a9fb7374e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:10.835265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91e9c-10e7-448f-bd90-1a9fb7374e20 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:10.835548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91e9c-10e7-448f-bd90-1a9fb7374e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:10.835819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91e9c-10e7-448f-bd90-1a9fb7374e20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:10.847458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a91e9c-10e7-448f-bd90-1a9fb7374e20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-69a91e9c-10e7-448f-bd90-1a9fb7374e20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:24:10.855704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:24:10.857473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:24:10.858499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:24:10.858499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:24:10.858499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:24:10.858499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:24:10.858499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:24:10.858499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:24:10.858499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:24:10.858499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:24:10.858499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:24:10.858499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:24:10.858499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:24:10.858499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:24:10.858499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 590/1180] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:24:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:24:10.863149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d95cd-f45d-48f8-9770-cd261c5bba4f None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:10.866327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:10.970419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d95cd-f45d-48f8-9770-cd261c5bba4f None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:11.006143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83b628-54ce-46e2-92fb-c441a7c6a15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:11.006812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83b628-54ce-46e2-92fb-c441a7c6a15e None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:11.007093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83b628-54ce-46e2-92fb-c441a7c6a15e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:11.007372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83b628-54ce-46e2-92fb-c441a7c6a15e None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:11.008222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83b628-54ce-46e2-92fb-c441a7c6a15e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:24:11.008548 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83b628-54ce-46e2-92fb-c441a7c6a15e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:11.010033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83b628-54ce-46e2-92fb-c441a7c6a15e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:11.018449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 591/1181] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:24:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:11.024923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a91e9c-10e7-448f-bd90-1a9fb7374e20 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:11.044534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:11.127438 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a91e9c-10e7-448f-bd90-1a9fb7374e20 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:11.160893 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bf21d-e1fd-4085-af3b-f1ba89785904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:11.161795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bf21d-e1fd-4085-af3b-f1ba89785904 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:11.162079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bf21d-e1fd-4085-af3b-f1ba89785904 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:11.162355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bf21d-e1fd-4085-af3b-f1ba89785904 None admin] PATH_INFO: `/v3/users/aabc51f0d8ac4d4b8b27a41fee9f018e` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:11.176205 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bf21d-e1fd-4085-af3b-f1ba89785904 None admin] RBAC: Authorizing `identity:get_user(user_id=aabc51f0d8ac4d4b8b27a41fee9f018e)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:11.177687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bf21d-e1fd-4085-af3b-f1ba89785904 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:11.179377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 591/1182] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:24:11 2022] GET /identity/v3/users/aabc51f0d8ac4d4b8b27a41fee9f018e => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:11.185509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83b628-54ce-46e2-92fb-c441a7c6a15e None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:11.251053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:11.286701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83b628-54ce-46e2-92fb-c441a7c6a15e None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:11.319932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9894a12-8990-4c8f-8ad6-98f403e8082d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:11.320781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9894a12-8990-4c8f-8ad6-98f403e8082d None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:11.321064 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9894a12-8990-4c8f-8ad6-98f403e8082d None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:11.321445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9894a12-8990-4c8f-8ad6-98f403e8082d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:11.387814 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9894a12-8990-4c8f-8ad6-98f403e8082d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:11.389218 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9894a12-8990-4c8f-8ad6-98f403e8082d None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:11.450833 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:11.486355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 592/1183] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:24:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:24:11.492430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bf21d-e1fd-4085-af3b-f1ba89785904 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:11.565360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:11.591667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bf21d-e1fd-4085-af3b-f1ba89785904 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:11.625092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26440e48-fa13-4df5-bd06-cc6dd2046f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:11.625960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26440e48-fa13-4df5-bd06-cc6dd2046f49 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:11.626239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26440e48-fa13-4df5-bd06-cc6dd2046f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:11.626511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26440e48-fa13-4df5-bd06-cc6dd2046f49 None admin] PATH_INFO: `/v3/projects/63504ba3b83944748ea65e891d57f256` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:11.627667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26440e48-fa13-4df5-bd06-cc6dd2046f49 None admin] RBAC: Authorizing `identity:get_project(project_id=63504ba3b83944748ea65e891d57f256)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:11.629074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26440e48-fa13-4df5-bd06-cc6dd2046f49 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:11.630786 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 592/1184] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:24:11 2022] GET /identity/v3/projects/63504ba3b83944748ea65e891d57f256 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:11.637677 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9894a12-8990-4c8f-8ad6-98f403e8082d None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:11.736808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9894a12-8990-4c8f-8ad6-98f403e8082d None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:11.771546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b172b6-c4b3-4952-bc9f-50eb13ab1ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:11.772201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b172b6-c4b3-4952-bc9f-50eb13ab1ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:11.772484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b172b6-c4b3-4952-bc9f-50eb13ab1ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:11.772759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b172b6-c4b3-4952-bc9f-50eb13ab1ba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:11.773654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b172b6-c4b3-4952-bc9f-50eb13ab1ba8 None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=63504ba3b83944748ea65e891d57f256, user.id=aabc51f0d8ac4d4b8b27a41fee9f018e) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:24:11.773958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b172b6-c4b3-4952-bc9f-50eb13ab1ba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:11.775395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b172b6-c4b3-4952-bc9f-50eb13ab1ba8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:11.780953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 593/1185] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:24:11 2022] GET /identity/v3/role_assignments?user.id=aabc51f0d8ac4d4b8b27a41fee9f018e&scope.project.id=63504ba3b83944748ea65e891d57f256&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:14.081957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170108f-fb9c-462f-ab73-57c47e951a30 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:14.082595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170108f-fb9c-462f-ab73-57c47e951a30 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:14.082595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170108f-fb9c-462f-ab73-57c47e951a30 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:14.083857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 593/1186] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:24:14.089677 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a13d9-4502-4cb3-9ccb-734f6ceb1c2e None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:14.090132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a13d9-4502-4cb3-9ccb-734f6ceb1c2e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:14.090298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a13d9-4502-4cb3-9ccb-734f6ceb1c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:14.106763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-462a13d9-4502-4cb3-9ccb-734f6ceb1c2e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:24:14.107979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-462a13d9-4502-4cb3-9ccb-734f6ceb1c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:24:14.133784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:14.187103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 594/1187] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:24:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:24:14.220254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170108f-fb9c-462f-ab73-57c47e951a30 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:14.355780 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170108f-fb9c-462f-ab73-57c47e951a30 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:14.403778 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f96a1-c125-4add-9c58-8768a71e16d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:14.404602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f96a1-c125-4add-9c58-8768a71e16d1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:14.404882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f96a1-c125-4add-9c58-8768a71e16d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:14.405188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f96a1-c125-4add-9c58-8768a71e16d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:14.416104 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f96a1-c125-4add-9c58-8768a71e16d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-c06f96a1-c125-4add-9c58-8768a71e16d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:24:14.424003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:24:14.425170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:24:14.430130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:24:14.430130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:24:14.430130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:24:14.430130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:24:14.430130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:24:14.430130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:24:14.430130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:24:14.430130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:24:14.430130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:24:14.430130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:24:14.430130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:24:14.430130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:24:14.430130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 594/1188] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:24:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:24:14.431400 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a13d9-4502-4cb3-9ccb-734f6ceb1c2e None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:14.557786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a13d9-4502-4cb3-9ccb-734f6ceb1c2e None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:14.605663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e5b53-1966-4637-999e-f458d062ea6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:14.606587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e5b53-1966-4637-999e-f458d062ea6d None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:14.606982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e5b53-1966-4637-999e-f458d062ea6d None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:14.607374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e5b53-1966-4637-999e-f458d062ea6d None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:14.608433 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e5b53-1966-4637-999e-f458d062ea6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:24:14.608871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e5b53-1966-4637-999e-f458d062ea6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:14.610871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e5b53-1966-4637-999e-f458d062ea6d None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:14.622140 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 595/1189] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:24:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:14.628118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f96a1-c125-4add-9c58-8768a71e16d1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:14.631013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:14.732503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f96a1-c125-4add-9c58-8768a71e16d1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:14.769794 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37311b-c131-4452-97b4-26a137c11774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:14.770669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37311b-c131-4452-97b4-26a137c11774 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:14.770952 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37311b-c131-4452-97b4-26a137c11774 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:14.771227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37311b-c131-4452-97b4-26a137c11774 None admin] PATH_INFO: `/v3/users/aabc51f0d8ac4d4b8b27a41fee9f018e` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:14.772412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37311b-c131-4452-97b4-26a137c11774 None admin] RBAC: Authorizing `identity:get_user(user_id=aabc51f0d8ac4d4b8b27a41fee9f018e)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:14.774188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37311b-c131-4452-97b4-26a137c11774 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:14.775831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 595/1190] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:24:14 2022] GET /identity/v3/users/aabc51f0d8ac4d4b8b27a41fee9f018e => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:14.781990 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e5b53-1966-4637-999e-f458d062ea6d None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:14.808872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:14.884249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e5b53-1966-4637-999e-f458d062ea6d None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:14.919948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7fb5f-38cb-4c27-b687-d0c65d609466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:14.920805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7fb5f-38cb-4c27-b687-d0c65d609466 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:14.921107 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7fb5f-38cb-4c27-b687-d0c65d609466 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:14.921385 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7fb5f-38cb-4c27-b687-d0c65d609466 None admin] PATH_INFO: `/v3/projects/63504ba3b83944748ea65e891d57f256` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:14.923517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7fb5f-38cb-4c27-b687-d0c65d609466 None admin] RBAC: Authorizing `identity:get_project(project_id=63504ba3b83944748ea65e891d57f256)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:14.924940 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7fb5f-38cb-4c27-b687-d0c65d609466 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:14.926665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 596/1191] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:24:14 2022] GET /identity/v3/projects/63504ba3b83944748ea65e891d57f256 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:14.933155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37311b-c131-4452-97b4-26a137c11774 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:15.001832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:15.032878 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37311b-c131-4452-97b4-26a137c11774 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:15.065860 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e6566-8343-4590-8e24-4c9377e8a202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:15.066665 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e6566-8343-4590-8e24-4c9377e8a202 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:15.066947 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e6566-8343-4590-8e24-4c9377e8a202 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:15.067222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e6566-8343-4590-8e24-4c9377e8a202 None admin] PATH_INFO: `/v3/projects/63504ba3b83944748ea65e891d57f256/users/aabc51f0d8ac4d4b8b27a41fee9f018e/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:15.069195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e6566-8343-4590-8e24-4c9377e8a202 None admin] RBAC: Authorizing `identity:create_grant(project_id=63504ba3b83944748ea65e891d57f256, user_id=aabc51f0d8ac4d4b8b27a41fee9f018e, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:15.071062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e6566-8343-4590-8e24-4c9377e8a202 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:15.084196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 596/1192] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:24:14 2022] PUT /identity/v3/projects/63504ba3b83944748ea65e891d57f256/users/aabc51f0d8ac4d4b8b27a41fee9f018e/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:24:17.370522 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57fafc-68c0-4b3c-8401-a385f66ab018 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:17.371033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57fafc-68c0-4b3c-8401-a385f66ab018 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:17.371130 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57fafc-68c0-4b3c-8401-a385f66ab018 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:17.372439 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 597/1193] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:24:17.377745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f1ca1-b3cc-4036-8075-549661f14ab8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:17.378185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f1ca1-b3cc-4036-8075-549661f14ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:17.378301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f1ca1-b3cc-4036-8075-549661f14ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:17.395111 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b09f1ca1-b3cc-4036-8075-549661f14ab8 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:24:17.396306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b09f1ca1-b3cc-4036-8075-549661f14ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:24:17.462076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 597/1194] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:24:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:24:17.489227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57fafc-68c0-4b3c-8401-a385f66ab018 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:17.582781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:17.594095 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57fafc-68c0-4b3c-8401-a385f66ab018 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:17.627996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c748cf-dac3-4ed4-ad65-8268e109e600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:17.628694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c748cf-dac3-4ed4-ad65-8268e109e600 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:17.628975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c748cf-dac3-4ed4-ad65-8268e109e600 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:17.629609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c748cf-dac3-4ed4-ad65-8268e109e600 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:17.641296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c748cf-dac3-4ed4-ad65-8268e109e600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-c4c748cf-dac3-4ed4-ad65-8268e109e600 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:24:17.650061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:24:17.651301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:24:17.652157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:24:17.652157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:24:17.652157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:24:17.652157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:24:17.652157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:24:17.652157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:24:17.652157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:24:17.652157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:24:17.652157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:24:17.652157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:24:17.652157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:24:17.652157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:24:17.652157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 598/1195] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:24:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:24:17.657549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f1ca1-b3cc-4036-8075-549661f14ab8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:17.717110 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:17.763918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f1ca1-b3cc-4036-8075-549661f14ab8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:17.798351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b9bc4-68e6-45b1-9b4b-d3f8c628ee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:17.799086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b9bc4-68e6-45b1-9b4b-d3f8c628ee32 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:17.799530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b9bc4-68e6-45b1-9b4b-d3f8c628ee32 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:17.799722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b9bc4-68e6-45b1-9b4b-d3f8c628ee32 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:17.800699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b9bc4-68e6-45b1-9b4b-d3f8c628ee32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:24:17.801054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b9bc4-68e6-45b1-9b4b-d3f8c628ee32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:17.802757 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b9bc4-68e6-45b1-9b4b-d3f8c628ee32 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:17.814513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 598/1196] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:24:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:17.822328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c748cf-dac3-4ed4-ad65-8268e109e600 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:17.924977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c748cf-dac3-4ed4-ad65-8268e109e600 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:17.959935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d966-55c8-40fa-986f-0c1b998dbd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:17.960875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d966-55c8-40fa-986f-0c1b998dbd83 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:17.961336 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d966-55c8-40fa-986f-0c1b998dbd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:17.961691 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d966-55c8-40fa-986f-0c1b998dbd83 None admin] PATH_INFO: `/v3/users/aabc51f0d8ac4d4b8b27a41fee9f018e` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:17.962795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6d966-55c8-40fa-986f-0c1b998dbd83 None admin] RBAC: Authorizing `identity:get_user(user_id=aabc51f0d8ac4d4b8b27a41fee9f018e)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:17.964220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6d966-55c8-40fa-986f-0c1b998dbd83 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:17.965861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 599/1197] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jun 7 08:24:17 2022] GET /identity/v3/users/aabc51f0d8ac4d4b8b27a41fee9f018e => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:17.971824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b9bc4-68e6-45b1-9b4b-d3f8c628ee32 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:18.076310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b9bc4-68e6-45b1-9b4b-d3f8c628ee32 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:18.098441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:18.112950 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c11ec0-29c2-439b-b7f9-f156fb7084e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:18.114170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c11ec0-29c2-439b-b7f9-f156fb7084e3 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:18.114170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c11ec0-29c2-439b-b7f9-f156fb7084e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:18.114170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c11ec0-29c2-439b-b7f9-f156fb7084e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:18.182556 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c11ec0-29c2-439b-b7f9-f156fb7084e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:18.183903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c11ec0-29c2-439b-b7f9-f156fb7084e3 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:18.281320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 599/1198] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:24:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:24:18.287347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d966-55c8-40fa-986f-0c1b998dbd83 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:18.307435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:18.387529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d966-55c8-40fa-986f-0c1b998dbd83 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:18.421827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3f3e0-caf1-489b-b974-4702a99e3625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:18.422672 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3f3e0-caf1-489b-b974-4702a99e3625 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:18.422952 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3f3e0-caf1-489b-b974-4702a99e3625 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:18.423223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3f3e0-caf1-489b-b974-4702a99e3625 None admin] PATH_INFO: `/v3/projects/63504ba3b83944748ea65e891d57f256` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:18.424383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3f3e0-caf1-489b-b974-4702a99e3625 None admin] RBAC: Authorizing `identity:get_project(project_id=63504ba3b83944748ea65e891d57f256)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:18.425820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3f3e0-caf1-489b-b974-4702a99e3625 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:18.427515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 600/1199] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jun 7 08:24:18 2022] GET /identity/v3/projects/63504ba3b83944748ea65e891d57f256 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:18.434412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c11ec0-29c2-439b-b7f9-f156fb7084e3 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:18.458977 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:18.534807 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c11ec0-29c2-439b-b7f9-f156fb7084e3 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:18.572565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5c375-e33d-41fc-87e9-209586caa32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:18.573281 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5c375-e33d-41fc-87e9-209586caa32d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:18.573569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5c375-e33d-41fc-87e9-209586caa32d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:18.573848 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5c375-e33d-41fc-87e9-209586caa32d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:18.574700 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5c375-e33d-41fc-87e9-209586caa32d None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=63504ba3b83944748ea65e891d57f256, user.id=aabc51f0d8ac4d4b8b27a41fee9f018e) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:24:18.575008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5c375-e33d-41fc-87e9-209586caa32d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:18.576462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5c375-e33d-41fc-87e9-209586caa32d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:18.583602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 600/1200] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:24:18 2022] GET /identity/v3/role_assignments?user.id=aabc51f0d8ac4d4b8b27a41fee9f018e&scope.project.id=63504ba3b83944748ea65e891d57f256&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:28.417618 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=57 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:28.580412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=57 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:21.062672 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d3900-6b8d-4a85-939e-243949bcc599 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:21.063289 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d3900-6b8d-4a85-939e-243949bcc599 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:21.063289 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d3900-6b8d-4a85-939e-243949bcc599 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:21.064939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 601/1201] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:25:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:25:21.071156 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc318c-4775-4312-894b-b72743ff3096 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:21.071694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc318c-4775-4312-894b-b72743ff3096 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:21.071802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc318c-4775-4312-894b-b72743ff3096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:21.095217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6afc318c-4775-4312-894b-b72743ff3096 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:25:21.096535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6afc318c-4775-4312-894b-b72743ff3096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:25:21.216728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 601/1202] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:25:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:25:21.245302 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d3900-6b8d-4a85-939e-243949bcc599 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:21.409627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d3900-6b8d-4a85-939e-243949bcc599 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:21.453671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb16d63-acf2-42de-8c82-a05f05e18602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:21.454818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb16d63-acf2-42de-8c82-a05f05e18602 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:21.455133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb16d63-acf2-42de-8c82-a05f05e18602 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:21.455451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb16d63-acf2-42de-8c82-a05f05e18602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:21.470281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb16d63-acf2-42de-8c82-a05f05e18602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:21.471835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb16d63-acf2-42de-8c82-a05f05e18602 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-3bb16d63-acf2-42de-8c82-a05f05e18602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:25:21.494113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:21.495243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:21.496626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:25:21.496626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:25:21.496626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:21.496626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:21.502227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc318c-4775-4312-894b-b72743ff3096 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:21.504478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 602/1203] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:21.633197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc318c-4775-4312-894b-b72743ff3096 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:21.668388 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:21.677755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf780b1c-a45d-4797-8719-3537c6881acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:21.678738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf780b1c-a45d-4797-8719-3537c6881acb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:21.679066 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf780b1c-a45d-4797-8719-3537c6881acb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:21.679399 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf780b1c-a45d-4797-8719-3537c6881acb None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:21.680152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf780b1c-a45d-4797-8719-3537c6881acb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:21.680462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf780b1c-a45d-4797-8719-3537c6881acb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:21.681963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf780b1c-a45d-4797-8719-3537c6881acb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:21.697388 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 602/1204] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:21.703137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb16d63-acf2-42de-8c82-a05f05e18602 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:21.777172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:21.839288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb16d63-acf2-42de-8c82-a05f05e18602 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:21.889062 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d53cca-73ae-4a60-aa63-e0b283224e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:21.889865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d53cca-73ae-4a60-aa63-e0b283224e28 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:21.890257 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d53cca-73ae-4a60-aa63-e0b283224e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:21.890624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d53cca-73ae-4a60-aa63-e0b283224e28 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:21.891372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d53cca-73ae-4a60-aa63-e0b283224e28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:21.895280 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d53cca-73ae-4a60-aa63-e0b283224e28 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:21.921341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 603/1205] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jun 7 08:25:21 2022] POST /identity/v3/users => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:25:24.516206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f6b88-b287-4c02-979a-b8ab15154caf None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:24.516787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f6b88-b287-4c02-979a-b8ab15154caf None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:24.516787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f6b88-b287-4c02-979a-b8ab15154caf None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:24.518175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 603/1206] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:25:24.524265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499cb62-3d37-4a22-9017-3a90180fb60d None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:24.524735 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499cb62-3d37-4a22-9017-3a90180fb60d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:24.524863 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499cb62-3d37-4a22-9017-3a90180fb60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:24.543635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-6499cb62-3d37-4a22-9017-3a90180fb60d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:25:24.545115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6499cb62-3d37-4a22-9017-3a90180fb60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:25:24.625492 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 604/1207] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:25:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:25:24.650008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f6b88-b287-4c02-979a-b8ab15154caf None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:24.817678 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f6b88-b287-4c02-979a-b8ab15154caf None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:24.860016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4426b48-bf9e-49de-92b3-1d6959bbe903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:24.861016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4426b48-bf9e-49de-92b3-1d6959bbe903 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:24.861346 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4426b48-bf9e-49de-92b3-1d6959bbe903 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:24.861751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4426b48-bf9e-49de-92b3-1d6959bbe903 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:24.876577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4426b48-bf9e-49de-92b3-1d6959bbe903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-d4426b48-bf9e-49de-92b3-1d6959bbe903 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:25:24.886942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:24.889412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:24.890633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:24.890633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:24.890633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:24.890633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:24.890633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:25:24.890633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:25:24.890633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:25:24.890633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:25:24.890633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:25:24.890633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:25:24.890633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:25:24.890633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:24.890633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 604/1208] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:25:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:24.897044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499cb62-3d37-4a22-9017-3a90180fb60d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:24.948533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:25.023378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499cb62-3d37-4a22-9017-3a90180fb60d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:25.066875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827cf83f-924b-457b-ab10-690d7dbc4221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:25.067741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cf83f-924b-457b-ab10-690d7dbc4221 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:25.068360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cf83f-924b-457b-ab10-690d7dbc4221 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:25.068645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cf83f-924b-457b-ab10-690d7dbc4221 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:25.069448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827cf83f-924b-457b-ab10-690d7dbc4221 None admin] RBAC: Adding query filter params (name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:25.069840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827cf83f-924b-457b-ab10-690d7dbc4221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:25.071636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827cf83f-924b-457b-ab10-690d7dbc4221 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:25.082794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 605/1209] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:25:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:25.089181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4426b48-bf9e-49de-92b3-1d6959bbe903 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:25.144994 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:25.211326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4426b48-bf9e-49de-92b3-1d6959bbe903 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:25.250021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4c67a-8801-420f-8fbb-cf1793581d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:25.250918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4c67a-8801-420f-8fbb-cf1793581d7b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:25.251276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4c67a-8801-420f-8fbb-cf1793581d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:25.253830 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4c67a-8801-420f-8fbb-cf1793581d7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:25.267782 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4c67a-8801-420f-8fbb-cf1793581d7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:25.269401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4c67a-8801-420f-8fbb-cf1793581d7b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-2fa4c67a-8801-420f-8fbb-cf1793581d7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:25:25.290667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:25.293904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:25.295017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:25:25.295017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:25:25.295017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:25.295017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:25.295017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 605/1210] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:25.299410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827cf83f-924b-457b-ab10-690d7dbc4221 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:25.422854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:25.425854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827cf83f-924b-457b-ab10-690d7dbc4221 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:25.469308 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09a098-d3e8-4eb4-90a8-e2d11f208e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:25.470015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09a098-d3e8-4eb4-90a8-e2d11f208e3f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:25.470307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09a098-d3e8-4eb4-90a8-e2d11f208e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:25.470624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09a098-d3e8-4eb4-90a8-e2d11f208e3f None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:25.471379 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09a098-d3e8-4eb4-90a8-e2d11f208e3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:25.471642 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09a098-d3e8-4eb4-90a8-e2d11f208e3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:25.473162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09a098-d3e8-4eb4-90a8-e2d11f208e3f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:25.486489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 606/1211] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:25:25.493314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4c67a-8801-420f-8fbb-cf1793581d7b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:25.614140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4c67a-8801-420f-8fbb-cf1793581d7b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:25.616691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:25.655664 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede1e9a-cbf3-48a0-9ba3-4084dc9c24f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:25.656471 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede1e9a-cbf3-48a0-9ba3-4084dc9c24f7 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:25.656750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede1e9a-cbf3-48a0-9ba3-4084dc9c24f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:25.657020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede1e9a-cbf3-48a0-9ba3-4084dc9c24f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:25.666311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede1e9a-cbf3-48a0-9ba3-4084dc9c24f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:25.667884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede1e9a-cbf3-48a0-9ba3-4084dc9c24f7 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-4ede1e9a-cbf3-48a0-9ba3-4084dc9c24f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:25:25.685575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:25.686749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:25.688227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:25.688227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:25.688227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:25:25.688227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:25:25.688227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:25:25.688227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:25:25.688227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:25:25.688227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:25:25.688227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:25:25.688227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:25.688227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 606/1212] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:25.692701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09a098-d3e8-4eb4-90a8-e2d11f208e3f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:25.815367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09a098-d3e8-4eb4-90a8-e2d11f208e3f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:25.865821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd457fe7-c420-4a1a-aae6-3714e6ada5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:25.866551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd457fe7-c420-4a1a-aae6-3714e6ada5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:25.866842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd457fe7-c420-4a1a-aae6-3714e6ada5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:25.867128 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd457fe7-c420-4a1a-aae6-3714e6ada5a8 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:25.868222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd457fe7-c420-4a1a-aae6-3714e6ada5a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:25.868571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd457fe7-c420-4a1a-aae6-3714e6ada5a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:25.870224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd457fe7-c420-4a1a-aae6-3714e6ada5a8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:25.890662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 607/1213] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Jun 7 08:25:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:25.899425 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede1e9a-cbf3-48a0-9ba3-4084dc9c24f7 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:26.024759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede1e9a-cbf3-48a0-9ba3-4084dc9c24f7 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:26.067975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e69e7-9cc4-49cb-9a09-2afda37a2fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:26.069284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e69e7-9cc4-49cb-9a09-2afda37a2fed None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:26.069572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e69e7-9cc4-49cb-9a09-2afda37a2fed None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:26.069869 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e69e7-9cc4-49cb-9a09-2afda37a2fed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:26.112296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:26.167687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e69e7-9cc4-49cb-9a09-2afda37a2fed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:26.169184 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e69e7-9cc4-49cb-9a09-2afda37a2fed None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:26.299479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 607/1214] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:25:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:25:26.305830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd457fe7-c420-4a1a-aae6-3714e6ada5a8 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:26.345282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:26.470080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd457fe7-c420-4a1a-aae6-3714e6ada5a8 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:26.519631 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09431473-f5dc-4b9b-b0d8-c4223c22729b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:26.520470 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09431473-f5dc-4b9b-b0d8-c4223c22729b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:26.520759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09431473-f5dc-4b9b-b0d8-c4223c22729b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:26.521629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09431473-f5dc-4b9b-b0d8-c4223c22729b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:26.536711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09431473-f5dc-4b9b-b0d8-c4223c22729b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:26.538403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09431473-f5dc-4b9b-b0d8-c4223c22729b None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-09431473-f5dc-4b9b-b0d8-c4223c22729b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:25:26.562394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:26.563511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:26.565490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:25:26.565490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:25:26.565490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:26.565490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:26.565490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 608/1215] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:26.570639 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e69e7-9cc4-49cb-9a09-2afda37a2fed None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:26.660949 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:26.721517 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e69e7-9cc4-49cb-9a09-2afda37a2fed None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:26.781200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a1bba-3c84-4049-a0a0-0c4ce3610c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:26.782022 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a1bba-3c84-4049-a0a0-0c4ce3610c7d None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:26.782436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a1bba-3c84-4049-a0a0-0c4ce3610c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:26.782976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a1bba-3c84-4049-a0a0-0c4ce3610c7d None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:26.783976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a1bba-3c84-4049-a0a0-0c4ce3610c7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:26.784278 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a1bba-3c84-4049-a0a0-0c4ce3610c7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:26.786164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a1bba-3c84-4049-a0a0-0c4ce3610c7d None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:26.822963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 608/1216] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:26.825141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09431473-f5dc-4b9b-b0d8-c4223c22729b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:26.955203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:26.996022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09431473-f5dc-4b9b-b0d8-c4223c22729b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:27.048251 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97988c-70b9-452b-bf83-ed41e00dc04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:27.049903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97988c-70b9-452b-bf83-ed41e00dc04c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:27.050192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97988c-70b9-452b-bf83-ed41e00dc04c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:27.050670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97988c-70b9-452b-bf83-ed41e00dc04c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:27.062994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97988c-70b9-452b-bf83-ed41e00dc04c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:27.064989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97988c-70b9-452b-bf83-ed41e00dc04c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-1c97988c-70b9-452b-bf83-ed41e00dc04c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:25:27.087472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:27.095479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:27.096567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:25:27.096567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:25:27.096567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:25:27.096567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:25:27.096567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:25:27.096567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:25:27.096567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:25:27.096567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:27.096567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 609/1217] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:25:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:27.099718 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a1bba-3c84-4049-a0a0-0c4ce3610c7d None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:27.250306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a1bba-3c84-4049-a0a0-0c4ce3610c7d None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:27.279270 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:27.303354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11f3a6-bd3c-42e7-b20b-d2c8095d6322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:27.304549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11f3a6-bd3c-42e7-b20b-d2c8095d6322 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:27.305011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11f3a6-bd3c-42e7-b20b-d2c8095d6322 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:27.305770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11f3a6-bd3c-42e7-b20b-d2c8095d6322 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:27.306727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11f3a6-bd3c-42e7-b20b-d2c8095d6322 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:27.307347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11f3a6-bd3c-42e7-b20b-d2c8095d6322 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:27.309384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11f3a6-bd3c-42e7-b20b-d2c8095d6322 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:27.337265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 609/1218] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:25:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:27.339652 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97988c-70b9-452b-bf83-ed41e00dc04c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:27.515187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97988c-70b9-452b-bf83-ed41e00dc04c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:27.549827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:27.558861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96bf70-0a11-4042-aab1-bd86150cbbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:27.559685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96bf70-0a11-4042-aab1-bd86150cbbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:27.560064 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96bf70-0a11-4042-aab1-bd86150cbbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:27.560468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96bf70-0a11-4042-aab1-bd86150cbbc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:27.561523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96bf70-0a11-4042-aab1-bd86150cbbc2 None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=7a7d1e8a0f4b4511a7dc8b27651744c8) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:27.561974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96bf70-0a11-4042-aab1-bd86150cbbc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:27.563528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96bf70-0a11-4042-aab1-bd86150cbbc2 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:27.574436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 610/1219] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:25:27 2022] GET /identity/v3/role_assignments?user.id=7a7d1e8a0f4b4511a7dc8b27651744c8&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:30.004010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acafd6-63c9-4139-9630-b79c38563022 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:30.004558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acafd6-63c9-4139-9630-b79c38563022 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:30.004558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acafd6-63c9-4139-9630-b79c38563022 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:30.006430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 610/1220] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:25:30.011904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768e141-1951-4a00-b067-1ac16f3c8db3 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:30.012391 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768e141-1951-4a00-b067-1ac16f3c8db3 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:30.012479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768e141-1951-4a00-b067-1ac16f3c8db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:30.030234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-9768e141-1951-4a00-b067-1ac16f3c8db3 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:25:30.031441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9768e141-1951-4a00-b067-1ac16f3c8db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:25:30.109323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 611/1221] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:25:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:25:30.133533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acafd6-63c9-4139-9630-b79c38563022 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:30.248096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acafd6-63c9-4139-9630-b79c38563022 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:30.288860 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101f3f6-bc94-4d3e-9f21-4db8a66b2768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:30.289764 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101f3f6-bc94-4d3e-9f21-4db8a66b2768 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:30.290112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101f3f6-bc94-4d3e-9f21-4db8a66b2768 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:30.290413 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101f3f6-bc94-4d3e-9f21-4db8a66b2768 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:30.304921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101f3f6-bc94-4d3e-9f21-4db8a66b2768 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-d101f3f6-bc94-4d3e-9f21-4db8a66b2768 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:25:30.314231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:30.315550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:30.318567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:30.318567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:30.318567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:30.318567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:30.318567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:25:30.318567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:25:30.318567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:25:30.318567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:25:30.318567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:25:30.318567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:25:30.318567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:25:30.318567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:30.318567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 611/1222] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:25:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:30.322321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768e141-1951-4a00-b067-1ac16f3c8db3 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:30.447297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768e141-1951-4a00-b067-1ac16f3c8db3 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:30.475813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:30.491203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380251f-43be-47de-af70-5763d00e94ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:30.492029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380251f-43be-47de-af70-5763d00e94ac None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:30.492486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380251f-43be-47de-af70-5763d00e94ac None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:30.492871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380251f-43be-47de-af70-5763d00e94ac None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:30.493775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380251f-43be-47de-af70-5763d00e94ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:30.494189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380251f-43be-47de-af70-5763d00e94ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:30.495795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380251f-43be-47de-af70-5763d00e94ac None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:30.509600 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 612/1223] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:25:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:30.512073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101f3f6-bc94-4d3e-9f21-4db8a66b2768 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:30.566828 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:30.651683 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101f3f6-bc94-4d3e-9f21-4db8a66b2768 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:30.696596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dc3c2-d181-4530-802d-298a0f97856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:30.697703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dc3c2-d181-4530-802d-298a0f97856a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:30.698042 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dc3c2-d181-4530-802d-298a0f97856a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:30.698437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dc3c2-d181-4530-802d-298a0f97856a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:30.711986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dc3c2-d181-4530-802d-298a0f97856a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:30.713724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dc3c2-d181-4530-802d-298a0f97856a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-971dc3c2-d181-4530-802d-298a0f97856a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:25:30.731912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:30.733255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:30.734389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:25:30.734389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:25:30.734389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:30.734389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:30.734389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 612/1224] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:25:30.739320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380251f-43be-47de-af70-5763d00e94ac None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:30.848144 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380251f-43be-47de-af70-5763d00e94ac None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:30.884811 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d589fd0-f94a-45b8-9ba6-96d144d81b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:30.885513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d589fd0-f94a-45b8-9ba6-96d144d81b27 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:30.885840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d589fd0-f94a-45b8-9ba6-96d144d81b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:30.886274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d589fd0-f94a-45b8-9ba6-96d144d81b27 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:30.887079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d589fd0-f94a-45b8-9ba6-96d144d81b27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:30.887461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d589fd0-f94a-45b8-9ba6-96d144d81b27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:30.889279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d589fd0-f94a-45b8-9ba6-96d144d81b27 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:30.913950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 613/1225] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:30.920178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dc3c2-d181-4530-802d-298a0f97856a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:31.045404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:31.055114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dc3c2-d181-4530-802d-298a0f97856a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:31.100212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f069b1-4d8b-49a8-a80b-5d14bef5e138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:31.100953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f069b1-4d8b-49a8-a80b-5d14bef5e138 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:31.101285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f069b1-4d8b-49a8-a80b-5d14bef5e138 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:31.101568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f069b1-4d8b-49a8-a80b-5d14bef5e138 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:31.109781 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f069b1-4d8b-49a8-a80b-5d14bef5e138 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:31.111244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f069b1-4d8b-49a8-a80b-5d14bef5e138 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-93f069b1-4d8b-49a8-a80b-5d14bef5e138 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:25:31.126908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:31.128837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:31.131183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:31.131183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:31.131183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:25:31.131183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:25:31.131183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:25:31.131183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:25:31.131183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:25:31.131183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:25:31.131183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:25:31.131183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:31.131183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 613/1226] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:31.135101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d589fd0-f94a-45b8-9ba6-96d144d81b27 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:31.192155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:31.261979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d589fd0-f94a-45b8-9ba6-96d144d81b27 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:31.300512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33770044-b082-493e-9b99-6cac4d11106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:31.301304 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33770044-b082-493e-9b99-6cac4d11106c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:31.301666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33770044-b082-493e-9b99-6cac4d11106c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:31.302581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33770044-b082-493e-9b99-6cac4d11106c None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:31.303671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33770044-b082-493e-9b99-6cac4d11106c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:31.304197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33770044-b082-493e-9b99-6cac4d11106c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:31.306503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33770044-b082-493e-9b99-6cac4d11106c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:31.340168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 614/1227] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Jun 7 08:25:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:31.346804 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f069b1-4d8b-49a8-a80b-5d14bef5e138 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:31.470638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f069b1-4d8b-49a8-a80b-5d14bef5e138 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:31.510429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ac0cb-727c-4fbd-b274-b35da1efa7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:31.511227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ac0cb-727c-4fbd-b274-b35da1efa7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:31.511514 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ac0cb-727c-4fbd-b274-b35da1efa7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:31.511790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ac0cb-727c-4fbd-b274-b35da1efa7b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:31.526540 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ac0cb-727c-4fbd-b274-b35da1efa7b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:31.529189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ac0cb-727c-4fbd-b274-b35da1efa7b1 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-fc1ac0cb-727c-4fbd-b274-b35da1efa7b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:25:31.559511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:31.560736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:31.561999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:25:31.561999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:25:31.561999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:31.561999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:31.561999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 614/1228] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:31.566750 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33770044-b082-493e-9b99-6cac4d11106c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:31.693005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33770044-b082-493e-9b99-6cac4d11106c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:31.696938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:31.742795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955da4a-bb13-4840-926f-13484b36047c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:31.743475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955da4a-bb13-4840-926f-13484b36047c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:31.743745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955da4a-bb13-4840-926f-13484b36047c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:31.744018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955da4a-bb13-4840-926f-13484b36047c None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:31.744775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955da4a-bb13-4840-926f-13484b36047c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:31.745100 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955da4a-bb13-4840-926f-13484b36047c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:31.746514 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955da4a-bb13-4840-926f-13484b36047c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:31.758938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 615/1229] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:31.764598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ac0cb-727c-4fbd-b274-b35da1efa7b1 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:31.774882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:31.886164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ac0cb-727c-4fbd-b274-b35da1efa7b1 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:31.930694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39e5b7-899f-41a1-8515-56b8f2aac091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:31.931506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39e5b7-899f-41a1-8515-56b8f2aac091 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:31.931794 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39e5b7-899f-41a1-8515-56b8f2aac091 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:31.932075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39e5b7-899f-41a1-8515-56b8f2aac091 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:31.940799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39e5b7-899f-41a1-8515-56b8f2aac091 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:31.942295 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39e5b7-899f-41a1-8515-56b8f2aac091 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-6a39e5b7-899f-41a1-8515-56b8f2aac091 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:25:31.957140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:31.958190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:31.959747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:25:31.959747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:25:31.959747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:25:31.959747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:25:31.959747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:25:31.959747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:25:31.959747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:25:31.959747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:31.959747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 615/1230] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:25:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:31.965643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955da4a-bb13-4840-926f-13484b36047c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:32.093254 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955da4a-bb13-4840-926f-13484b36047c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:32.137652 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661fd73-3aa3-4032-acf0-f287229364ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:32.138349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661fd73-3aa3-4032-acf0-f287229364ee None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:32.138639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661fd73-3aa3-4032-acf0-f287229364ee None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:32.139064 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661fd73-3aa3-4032-acf0-f287229364ee None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:32.140083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661fd73-3aa3-4032-acf0-f287229364ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:32.140393 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661fd73-3aa3-4032-acf0-f287229364ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:32.142173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661fd73-3aa3-4032-acf0-f287229364ee None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:32.158200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 616/1231] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:25:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:32.166365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39e5b7-899f-41a1-8515-56b8f2aac091 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:32.239409 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:32.276753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39e5b7-899f-41a1-8515-56b8f2aac091 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:32.317302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cee7f-3a08-46fa-9786-ab23f1bc7036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:32.318348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cee7f-3a08-46fa-9786-ab23f1bc7036 None admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:32.318801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cee7f-3a08-46fa-9786-ab23f1bc7036 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:32.319125 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cee7f-3a08-46fa-9786-ab23f1bc7036 None admin] PATH_INFO: `/v3/projects/07b861f95bf3498d990785ba72474d82/users/7a7d1e8a0f4b4511a7dc8b27651744c8/roles/9f07c51b3716428b89a49a756664305a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:32.342700 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cee7f-3a08-46fa-9786-ab23f1bc7036 None admin] RBAC: Authorizing `identity:create_grant(project_id=07b861f95bf3498d990785ba72474d82, user_id=7a7d1e8a0f4b4511a7dc8b27651744c8, role_id=9f07c51b3716428b89a49a756664305a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:32.344204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cee7f-3a08-46fa-9786-ab23f1bc7036 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:32.359832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 616/1232] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:25:32 2022] PUT /identity/v3/projects/07b861f95bf3498d990785ba72474d82/users/7a7d1e8a0f4b4511a7dc8b27651744c8/roles/9f07c51b3716428b89a49a756664305a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:25:34.863434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f19686-9072-4bbf-9b0c-2abe27a5b4d6 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:34.864073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f19686-9072-4bbf-9b0c-2abe27a5b4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:34.864173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f19686-9072-4bbf-9b0c-2abe27a5b4d6 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:34.866600 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 617/1233] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:25:34.872745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb8a05-c98d-4340-978f-84f55162ac3c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:34.873288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb8a05-c98d-4340-978f-84f55162ac3c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:34.873421 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb8a05-c98d-4340-978f-84f55162ac3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:34.891061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3cdb8a05-c98d-4340-978f-84f55162ac3c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:25:34.892649 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3cdb8a05-c98d-4340-978f-84f55162ac3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:25:34.972229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 617/1234] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:25:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:25:34.997765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f19686-9072-4bbf-9b0c-2abe27a5b4d6 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:35.028783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:35.114075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f19686-9072-4bbf-9b0c-2abe27a5b4d6 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:35.153714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02069ff3-6ae2-47bb-94fa-5d027552ba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:35.154519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02069ff3-6ae2-47bb-94fa-5d027552ba4b None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:35.154803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02069ff3-6ae2-47bb-94fa-5d027552ba4b None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:35.155078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02069ff3-6ae2-47bb-94fa-5d027552ba4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:35.166327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02069ff3-6ae2-47bb-94fa-5d027552ba4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-02069ff3-6ae2-47bb-94fa-5d027552ba4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:25:35.174543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:35.175858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:35.176788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:35.176788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:35.176788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:35.176788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:35.176788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:25:35.176788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:25:35.176788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:25:35.176788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:25:35.176788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:25:35.176788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:25:35.176788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:25:35.176788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:35.176788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 618/1235] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jun 7 08:25:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:35.182118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb8a05-c98d-4340-978f-84f55162ac3c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:35.232836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:35.296853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb8a05-c98d-4340-978f-84f55162ac3c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:35.340916 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a8829-d140-4123-8689-912ec1481c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:35.341762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a8829-d140-4123-8689-912ec1481c6a None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:35.342072 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a8829-d140-4123-8689-912ec1481c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:35.342355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a8829-d140-4123-8689-912ec1481c6a None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:35.343219 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a8829-d140-4123-8689-912ec1481c6a None admin] RBAC: Adding query filter params (name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:35.343587 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a8829-d140-4123-8689-912ec1481c6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:35.345175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a8829-d140-4123-8689-912ec1481c6a None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:35.357199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 618/1236] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jun 7 08:25:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:35.362823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02069ff3-6ae2-47bb-94fa-5d027552ba4b None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:35.468891 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:35.487822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02069ff3-6ae2-47bb-94fa-5d027552ba4b None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:35.525808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7997b-ebd3-4563-8d3d-a7a18c84bf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:35.526711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7997b-ebd3-4563-8d3d-a7a18c84bf89 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:35.526973 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7997b-ebd3-4563-8d3d-a7a18c84bf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:35.527249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7997b-ebd3-4563-8d3d-a7a18c84bf89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:35.539956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7997b-ebd3-4563-8d3d-a7a18c84bf89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:35.541401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7997b-ebd3-4563-8d3d-a7a18c84bf89 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-53b7997b-ebd3-4563-8d3d-a7a18c84bf89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:25:35.561832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:35.563263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:35.564343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:25:35.564343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:25:35.564343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:35.564343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:35.564343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 619/1237] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:35.571921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a8829-d140-4123-8689-912ec1481c6a None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:35.699910 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a8829-d140-4123-8689-912ec1481c6a None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:35.710867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:35.740170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ec37c-41ea-4a11-8751-6a235065de45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:35.740807 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ec37c-41ea-4a11-8751-6a235065de45 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:35.741194 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ec37c-41ea-4a11-8751-6a235065de45 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:35.741503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ec37c-41ea-4a11-8751-6a235065de45 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:35.742237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ec37c-41ea-4a11-8751-6a235065de45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:35.742548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ec37c-41ea-4a11-8751-6a235065de45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:35.744175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ec37c-41ea-4a11-8751-6a235065de45 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:35.758442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 619/1238] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:35.764637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7997b-ebd3-4563-8d3d-a7a18c84bf89 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:35.885499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7997b-ebd3-4563-8d3d-a7a18c84bf89 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:35.928733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb53331-a043-4e1b-89fd-7eef6027a257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:35.929710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb53331-a043-4e1b-89fd-7eef6027a257 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:35.930104 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb53331-a043-4e1b-89fd-7eef6027a257 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:35.930421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb53331-a043-4e1b-89fd-7eef6027a257 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:35.933986 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:35.940783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb53331-a043-4e1b-89fd-7eef6027a257 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:35.942894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb53331-a043-4e1b-89fd-7eef6027a257 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-3bb53331-a043-4e1b-89fd-7eef6027a257 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:25:35.962275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:35.963325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:35.964297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:35.964297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:35.964297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:25:35.964297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:25:35.964297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:25:35.964297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:25:35.964297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:25:35.964297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:25:35.964297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:25:35.964297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:35.964297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 620/1239] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:35.969947 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ec37c-41ea-4a11-8751-6a235065de45 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:36.088109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ec37c-41ea-4a11-8751-6a235065de45 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:36.127164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7e4ea-cdb7-4335-9246-1572681ec212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:36.127846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7e4ea-cdb7-4335-9246-1572681ec212 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:36.128130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7e4ea-cdb7-4335-9246-1572681ec212 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:36.128455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7e4ea-cdb7-4335-9246-1572681ec212 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:36.129317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7e4ea-cdb7-4335-9246-1572681ec212 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:36.129639 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7e4ea-cdb7-4335-9246-1572681ec212 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:36.131633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7e4ea-cdb7-4335-9246-1572681ec212 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:36.154829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 620/1240] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Jun 7 08:25:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:36.160876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb53331-a043-4e1b-89fd-7eef6027a257 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:36.293889 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb53331-a043-4e1b-89fd-7eef6027a257 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:36.341656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c33376-e11e-4b69-b0c0-59b602923f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:36.342241 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c33376-e11e-4b69-b0c0-59b602923f3f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:36.342526 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c33376-e11e-4b69-b0c0-59b602923f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:36.342832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c33376-e11e-4b69-b0c0-59b602923f3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:36.419233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:36.427635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c33376-e11e-4b69-b0c0-59b602923f3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:36.429278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c33376-e11e-4b69-b0c0-59b602923f3f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:36.561246 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 621/1241] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:25:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 08:25:36.567378 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7e4ea-cdb7-4335-9246-1572681ec212 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:36.599716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:36.681707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7e4ea-cdb7-4335-9246-1572681ec212 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:36.718564 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dea27c-1db5-4c82-9fe4-c6c04dfde6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:36.724701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dea27c-1db5-4c82-9fe4-c6c04dfde6fb None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:36.725013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dea27c-1db5-4c82-9fe4-c6c04dfde6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:36.725323 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dea27c-1db5-4c82-9fe4-c6c04dfde6fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:36.738140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dea27c-1db5-4c82-9fe4-c6c04dfde6fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:36.740404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dea27c-1db5-4c82-9fe4-c6c04dfde6fb None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-70dea27c-1db5-4c82-9fe4-c6c04dfde6fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:25:36.769102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:36.770365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:36.771250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:25:36.771250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:25:36.771250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:36.771250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:36.771250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 621/1242] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:36.776651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c33376-e11e-4b69-b0c0-59b602923f3f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:36.883056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:36.889292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c33376-e11e-4b69-b0c0-59b602923f3f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:36.927556 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d203b-ba54-49d3-8ed8-edf174915247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:36.928237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d203b-ba54-49d3-8ed8-edf174915247 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:36.928520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d203b-ba54-49d3-8ed8-edf174915247 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:36.928796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d203b-ba54-49d3-8ed8-edf174915247 None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:36.929792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d203b-ba54-49d3-8ed8-edf174915247 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:36.930100 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d203b-ba54-49d3-8ed8-edf174915247 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:36.931529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d203b-ba54-49d3-8ed8-edf174915247 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:36.945896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 622/1243] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:36.951524 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dea27c-1db5-4c82-9fe4-c6c04dfde6fb None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:37.045332 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:37.069243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dea27c-1db5-4c82-9fe4-c6c04dfde6fb None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:37.112426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacdb4c9-6bf5-4c36-bf70-6799d1d7e8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:37.113581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdb4c9-6bf5-4c36-bf70-6799d1d7e8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:37.113984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdb4c9-6bf5-4c36-bf70-6799d1d7e8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:37.114276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdb4c9-6bf5-4c36-bf70-6799d1d7e8a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:37.123392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacdb4c9-6bf5-4c36-bf70-6799d1d7e8a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:37.125137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacdb4c9-6bf5-4c36-bf70-6799d1d7e8a0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-bacdb4c9-6bf5-4c36-bf70-6799d1d7e8a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:25:37.140147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:37.141433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:37.142522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:25:37.142522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:25:37.142522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:25:37.142522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:25:37.142522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:25:37.142522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:25:37.142522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:25:37.142522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:37.142522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 622/1244] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:25:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:37.147629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d203b-ba54-49d3-8ed8-edf174915247 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:37.268103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d203b-ba54-49d3-8ed8-edf174915247 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:37.310887 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46c970-2452-4732-866d-aa2f70d3c332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:37.311743 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46c970-2452-4732-866d-aa2f70d3c332 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:37.312095 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46c970-2452-4732-866d-aa2f70d3c332 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:37.312421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46c970-2452-4732-866d-aa2f70d3c332 None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:37.313549 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46c970-2452-4732-866d-aa2f70d3c332 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:37.313971 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46c970-2452-4732-866d-aa2f70d3c332 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:37.315765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46c970-2452-4732-866d-aa2f70d3c332 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:37.331844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 623/1245] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:37.339638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacdb4c9-6bf5-4c36-bf70-6799d1d7e8a0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:37.463815 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacdb4c9-6bf5-4c36-bf70-6799d1d7e8a0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:37.503293 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb52242-5680-4175-aa08-04506c1552f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:37.504055 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb52242-5680-4175-aa08-04506c1552f2 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:37.504339 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb52242-5680-4175-aa08-04506c1552f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:37.504612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb52242-5680-4175-aa08-04506c1552f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:37.505558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb52242-5680-4175-aa08-04506c1552f2 None admin] RBAC: Adding query filter params (role.id=9f07c51b3716428b89a49a756664305a, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=7a7d1e8a0f4b4511a7dc8b27651744c8) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:37.505874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb52242-5680-4175-aa08-04506c1552f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:37.507441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb52242-5680-4175-aa08-04506c1552f2 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:37.514513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 623/1246] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:25:37 2022] GET /identity/v3/role_assignments?user.id=7a7d1e8a0f4b4511a7dc8b27651744c8&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=9f07c51b3716428b89a49a756664305a => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:40.083869 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcadc6-8e90-4a05-8992-b09fd5d9882e None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:40.084525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcadc6-8e90-4a05-8992-b09fd5d9882e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:40.084525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcadc6-8e90-4a05-8992-b09fd5d9882e None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:40.085885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 624/1247] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:25:40.091623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e639a-27d7-4b6d-90f3-efd5671de13d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:40.092221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e639a-27d7-4b6d-90f3-efd5671de13d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:40.092221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e639a-27d7-4b6d-90f3-efd5671de13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:40.097383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:40.111644 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-702e639a-27d7-4b6d-90f3-efd5671de13d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:25:40.112962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-702e639a-27d7-4b6d-90f3-efd5671de13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:25:40.193198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 624/1248] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:25:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:25:40.218593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcadc6-8e90-4a05-8992-b09fd5d9882e None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:40.242846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:40.353718 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcadc6-8e90-4a05-8992-b09fd5d9882e None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:40.396310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bd0ed-f424-4dfb-b02f-5539b24c2e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:40.397222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bd0ed-f424-4dfb-b02f-5539b24c2e3a None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:40.397476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bd0ed-f424-4dfb-b02f-5539b24c2e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:40.397793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bd0ed-f424-4dfb-b02f-5539b24c2e3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:40.409255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bd0ed-f424-4dfb-b02f-5539b24c2e3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-588bd0ed-f424-4dfb-b02f-5539b24c2e3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:25:40.417739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:40.418899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:40.419810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:40.419810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:40.419810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:40.419810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:40.419810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:25:40.419810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:25:40.419810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:25:40.419810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:25:40.419810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:25:40.419810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:25:40.419810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:25:40.419810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:40.419810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 625/1249] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:25:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:40.425523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e639a-27d7-4b6d-90f3-efd5671de13d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:40.545568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e639a-27d7-4b6d-90f3-efd5671de13d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:40.584110 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:40.590457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090504ed-7494-4215-b892-f8a8acd5e590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:40.591239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090504ed-7494-4215-b892-f8a8acd5e590 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:40.591534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090504ed-7494-4215-b892-f8a8acd5e590 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:40.591840 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090504ed-7494-4215-b892-f8a8acd5e590 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:40.592646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090504ed-7494-4215-b892-f8a8acd5e590 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:40.592962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090504ed-7494-4215-b892-f8a8acd5e590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:40.594617 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090504ed-7494-4215-b892-f8a8acd5e590 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:40.605128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 625/1250] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:25:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:40.610441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bd0ed-f424-4dfb-b02f-5539b24c2e3a None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:40.698928 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:40.726979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bd0ed-f424-4dfb-b02f-5539b24c2e3a None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:40.768597 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9bd0d-1e71-4737-b5ee-acaff049b573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:40.769523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9bd0d-1e71-4737-b5ee-acaff049b573 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:40.769833 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9bd0d-1e71-4737-b5ee-acaff049b573 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:40.770136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9bd0d-1e71-4737-b5ee-acaff049b573 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:40.783495 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9bd0d-1e71-4737-b5ee-acaff049b573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:40.785002 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9bd0d-1e71-4737-b5ee-acaff049b573 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-baa9bd0d-1e71-4737-b5ee-acaff049b573 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:25:40.806690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:40.807736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:40.808658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:25:40.808658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:25:40.808658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:40.808658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:40.810107 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 626/1251] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:40.816224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090504ed-7494-4215-b892-f8a8acd5e590 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:40.950682 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090504ed-7494-4215-b892-f8a8acd5e590 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:40.987161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6fdba-402c-4330-8c36-dfd1f1cd33a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:40.987943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6fdba-402c-4330-8c36-dfd1f1cd33a8 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:40.988232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6fdba-402c-4330-8c36-dfd1f1cd33a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:40.988513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6fdba-402c-4330-8c36-dfd1f1cd33a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:40.989380 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6fdba-402c-4330-8c36-dfd1f1cd33a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:40.989768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6fdba-402c-4330-8c36-dfd1f1cd33a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:40.991730 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6fdba-402c-4330-8c36-dfd1f1cd33a8 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:41.006500 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 626/1252] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:41.012587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9bd0d-1e71-4737-b5ee-acaff049b573 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:41.132220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9bd0d-1e71-4737-b5ee-acaff049b573 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:41.164952 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:41.173860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d7a70-113f-432d-b590-75fe793373ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:41.174669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d7a70-113f-432d-b590-75fe793373ee None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:41.174963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d7a70-113f-432d-b590-75fe793373ee None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:41.175224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d7a70-113f-432d-b590-75fe793373ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:41.183674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d7a70-113f-432d-b590-75fe793373ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:41.185161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d7a70-113f-432d-b590-75fe793373ee None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-4f6d7a70-113f-432d-b590-75fe793373ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:25:41.201804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:41.203179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:41.204789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:41.204789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:41.204789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:25:41.204789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:25:41.204789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:25:41.204789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:25:41.204789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:25:41.204789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:25:41.204789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:25:41.204789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:41.204789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 627/1253] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:41.210330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6fdba-402c-4330-8c36-dfd1f1cd33a8 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:41.276091 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:41.328881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6fdba-402c-4330-8c36-dfd1f1cd33a8 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:41.372254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178dc4c7-146b-47d2-a1bc-1b658f93cef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:41.372953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dc4c7-146b-47d2-a1bc-1b658f93cef4 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:41.373321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dc4c7-146b-47d2-a1bc-1b658f93cef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:41.373599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dc4c7-146b-47d2-a1bc-1b658f93cef4 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:41.374451 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dc4c7-146b-47d2-a1bc-1b658f93cef4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:41.374756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dc4c7-146b-47d2-a1bc-1b658f93cef4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:41.376244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dc4c7-146b-47d2-a1bc-1b658f93cef4 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:41.400054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 627/1254] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Jun 7 08:25:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:41.401571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d7a70-113f-432d-b590-75fe793373ee None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:41.517300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d7a70-113f-432d-b590-75fe793373ee None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:41.556599 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ac2df-51af-43dc-a382-c50c49da952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:41.557385 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ac2df-51af-43dc-a382-c50c49da952f None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:41.557676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ac2df-51af-43dc-a382-c50c49da952f None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:41.557980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ac2df-51af-43dc-a382-c50c49da952f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:41.620260 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:41.634620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ac2df-51af-43dc-a382-c50c49da952f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:41.636196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ac2df-51af-43dc-a382-c50c49da952f None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:41.761250 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 628/1255] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:25:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:25:41.766983 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178dc4c7-146b-47d2-a1bc-1b658f93cef4 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:41.882636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178dc4c7-146b-47d2-a1bc-1b658f93cef4 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:41.900631 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:41.925189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d1949-374b-48a7-a46d-4aa89f1c8b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:41.926315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1949-374b-48a7-a46d-4aa89f1c8b65 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:41.926598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1949-374b-48a7-a46d-4aa89f1c8b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:41.926932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1949-374b-48a7-a46d-4aa89f1c8b65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:41.937871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d1949-374b-48a7-a46d-4aa89f1c8b65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:41.939332 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d1949-374b-48a7-a46d-4aa89f1c8b65 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-103d1949-374b-48a7-a46d-4aa89f1c8b65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:25:41.958646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:41.960717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:41.962101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:25:41.962101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:25:41.962101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:41.962101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:41.965327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 628/1256] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:41.966296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ac2df-51af-43dc-a382-c50c49da952f None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:42.065115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:42.079603 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ac2df-51af-43dc-a382-c50c49da952f None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:42.125189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39cece5-5900-4e7b-b200-b3c74742774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:42.126039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cece5-5900-4e7b-b200-b3c74742774c None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:42.126387 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cece5-5900-4e7b-b200-b3c74742774c None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:42.126601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cece5-5900-4e7b-b200-b3c74742774c None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:42.128358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39cece5-5900-4e7b-b200-b3c74742774c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:42.128358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39cece5-5900-4e7b-b200-b3c74742774c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:42.129462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39cece5-5900-4e7b-b200-b3c74742774c None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:42.143519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 629/1257] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:25:42.150243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d1949-374b-48a7-a46d-4aa89f1c8b65 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:42.263886 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d1949-374b-48a7-a46d-4aa89f1c8b65 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:42.307987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caba34d-d4d0-4976-a86a-e02359b3eab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:42.309290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caba34d-d4d0-4976-a86a-e02359b3eab0 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:42.309646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caba34d-d4d0-4976-a86a-e02359b3eab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:42.309978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caba34d-d4d0-4976-a86a-e02359b3eab0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:42.322266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caba34d-d4d0-4976-a86a-e02359b3eab0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:42.324584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caba34d-d4d0-4976-a86a-e02359b3eab0 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-2caba34d-d4d0-4976-a86a-e02359b3eab0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:25:42.347084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:42.350655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:42.352366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:25:42.352366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:25:42.352366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:25:42.352366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:25:42.352366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:25:42.352366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:25:42.352366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:25:42.352366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:42.352366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 629/1258] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:42.357426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39cece5-5900-4e7b-b200-b3c74742774c None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:42.482846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39cece5-5900-4e7b-b200-b3c74742774c None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:42.521418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5a359-97da-4646-b7d0-0a71e84dc07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:42.522104 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5a359-97da-4646-b7d0-0a71e84dc07d None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:42.522392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5a359-97da-4646-b7d0-0a71e84dc07d None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:42.522671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5a359-97da-4646-b7d0-0a71e84dc07d None admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:42.523483 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5a359-97da-4646-b7d0-0a71e84dc07d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:42.523794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5a359-97da-4646-b7d0-0a71e84dc07d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:42.525317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5a359-97da-4646-b7d0-0a71e84dc07d None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:42.539606 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 630/1259] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:25:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:42.545904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caba34d-d4d0-4976-a86a-e02359b3eab0 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:42.575095 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:42.656515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caba34d-d4d0-4976-a86a-e02359b3eab0 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:42.694994 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96784feb-f6c4-40aa-b576-4c44afc496b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:42.695660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96784feb-f6c4-40aa-b576-4c44afc496b5 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:42.695941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96784feb-f6c4-40aa-b576-4c44afc496b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:42.696214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96784feb-f6c4-40aa-b576-4c44afc496b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:42.697051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96784feb-f6c4-40aa-b576-4c44afc496b5 None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=7a7d1e8a0f4b4511a7dc8b27651744c8) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:42.697387 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96784feb-f6c4-40aa-b576-4c44afc496b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:42.698828 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96784feb-f6c4-40aa-b576-4c44afc496b5 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:42.705605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 630/1260] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:25:42 2022] GET /identity/v3/role_assignments?user.id=7a7d1e8a0f4b4511a7dc8b27651744c8&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:44.990546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00631a-b234-4c1c-8eaf-ee78b68cffda None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:44.991102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00631a-b234-4c1c-8eaf-ee78b68cffda None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:44.991102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00631a-b234-4c1c-8eaf-ee78b68cffda None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:44.992459 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 631/1261] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:25:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:25:44.997754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d0f34-2b12-4c2a-83fb-d1063c01c6b4 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:44.998169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d0f34-2b12-4c2a-83fb-d1063c01c6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:44.998313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d0f34-2b12-4c2a-83fb-d1063c01c6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:45.016250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5b2d0f34-2b12-4c2a-83fb-d1063c01c6b4 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:25:45.017497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5b2d0f34-2b12-4c2a-83fb-d1063c01c6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:25:45.086329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 631/1262] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:25:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:25:45.110103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00631a-b234-4c1c-8eaf-ee78b68cffda None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:45.122101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:45.213504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00631a-b234-4c1c-8eaf-ee78b68cffda None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:45.256154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e182fa3-8018-4627-a487-700c9ab8cfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:45.256944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e182fa3-8018-4627-a487-700c9ab8cfdd None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:45.257258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e182fa3-8018-4627-a487-700c9ab8cfdd None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:45.257530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e182fa3-8018-4627-a487-700c9ab8cfdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:45.267801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e182fa3-8018-4627-a487-700c9ab8cfdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-6e182fa3-8018-4627-a487-700c9ab8cfdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:25:45.275673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:45.276792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:45.277855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:45.277855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:45.277855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:45.277855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:45.277855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:25:45.277855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:25:45.277855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:25:45.277855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:25:45.277855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:25:45.277855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:25:45.277855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:25:45.277855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:45.277855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 632/1263] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:25:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:25:45.282933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d0f34-2b12-4c2a-83fb-d1063c01c6b4 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:45.305172 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:45.396989 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d0f34-2b12-4c2a-83fb-d1063c01c6b4 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:45.432087 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1d9bf-09c2-40f0-b64a-c7774f7e6e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:45.432756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d9bf-09c2-40f0-b64a-c7774f7e6e42 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:45.433043 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d9bf-09c2-40f0-b64a-c7774f7e6e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:45.433347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d9bf-09c2-40f0-b64a-c7774f7e6e42 None admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:45.434094 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1d9bf-09c2-40f0-b64a-c7774f7e6e42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:45.434404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1d9bf-09c2-40f0-b64a-c7774f7e6e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:45.435786 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1d9bf-09c2-40f0-b64a-c7774f7e6e42 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:45.444413 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 632/1264] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:25:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:45.450266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e182fa3-8018-4627-a487-700c9ab8cfdd None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:45.518408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:45.553193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e182fa3-8018-4627-a487-700c9ab8cfdd None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:45.586788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2ee74-ba4c-4b5f-bac6-8681f2676516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:45.587578 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2ee74-ba4c-4b5f-bac6-8681f2676516 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:45.587864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2ee74-ba4c-4b5f-bac6-8681f2676516 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:45.588143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2ee74-ba4c-4b5f-bac6-8681f2676516 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:45.599419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2ee74-ba4c-4b5f-bac6-8681f2676516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:45.600877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2ee74-ba4c-4b5f-bac6-8681f2676516 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-71a2ee74-ba4c-4b5f-bac6-8681f2676516 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:25:45.618409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:45.619457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:45.620552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:25:45.620552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:25:45.620552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:45.620552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:45.620552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 633/1265] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:45.626850 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1d9bf-09c2-40f0-b64a-c7774f7e6e42 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:45.710296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:45.729864 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1d9bf-09c2-40f0-b64a-c7774f7e6e42 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:45.761962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44830f31-9ad9-439e-9c07-70edeb446e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:45.762622 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44830f31-9ad9-439e-9c07-70edeb446e15 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:45.762910 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44830f31-9ad9-439e-9c07-70edeb446e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:45.763187 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44830f31-9ad9-439e-9c07-70edeb446e15 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:45.763901 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44830f31-9ad9-439e-9c07-70edeb446e15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:45.764208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44830f31-9ad9-439e-9c07-70edeb446e15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:45.765595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44830f31-9ad9-439e-9c07-70edeb446e15 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:45.777579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 633/1266] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:45.783141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2ee74-ba4c-4b5f-bac6-8681f2676516 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:45.884519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2ee74-ba4c-4b5f-bac6-8681f2676516 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:45.905542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:45.919980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445545c-b808-475c-b59f-78f0bd68f53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:45.920771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445545c-b808-475c-b59f-78f0bd68f53a None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:45.921052 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445545c-b808-475c-b59f-78f0bd68f53a None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:45.921353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445545c-b808-475c-b59f-78f0bd68f53a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:45.929061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445545c-b808-475c-b59f-78f0bd68f53a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:45.930535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445545c-b808-475c-b59f-78f0bd68f53a None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-8445545c-b808-475c-b59f-78f0bd68f53a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:25:45.944982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:45.946206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:45.947552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:45.947552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:45.947552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:25:45.947552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:25:45.947552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:25:45.947552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:25:45.947552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:25:45.947552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:25:45.947552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:25:45.947552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:45.947552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 634/1267] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:45.951932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44830f31-9ad9-439e-9c07-70edeb446e15 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:46.051693 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44830f31-9ad9-439e-9c07-70edeb446e15 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:46.085132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71474b9b-1532-42b0-9437-c57796f08016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:46.085790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71474b9b-1532-42b0-9437-c57796f08016 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:46.086068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71474b9b-1532-42b0-9437-c57796f08016 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:46.086337 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71474b9b-1532-42b0-9437-c57796f08016 None admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:46.087155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71474b9b-1532-42b0-9437-c57796f08016 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:46.087459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71474b9b-1532-42b0-9437-c57796f08016 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:46.088853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71474b9b-1532-42b0-9437-c57796f08016 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:46.097267 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:46.106410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 634/1268] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Jun 7 08:25:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:46.111963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445545c-b808-475c-b59f-78f0bd68f53a None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:46.215567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445545c-b808-475c-b59f-78f0bd68f53a None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:46.248970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e408df-1586-40bd-8410-860b5fed2508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:46.249777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e408df-1586-40bd-8410-860b5fed2508 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:46.250057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e408df-1586-40bd-8410-860b5fed2508 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:46.250334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e408df-1586-40bd-8410-860b5fed2508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:46.261410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e408df-1586-40bd-8410-860b5fed2508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:46.262872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e408df-1586-40bd-8410-860b5fed2508 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-52e408df-1586-40bd-8410-860b5fed2508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:25:46.279485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:46.280558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:46.281875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:25:46.281875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:25:46.281875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:46.281875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:46.281875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 635/1269] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:46.286709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71474b9b-1532-42b0-9437-c57796f08016 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:46.399554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71474b9b-1532-42b0-9437-c57796f08016 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:46.436590 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8b4d8-d0bf-4369-bb81-5d6c60b37492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:46.437285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8b4d8-d0bf-4369-bb81-5d6c60b37492 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:46.437563 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8b4d8-d0bf-4369-bb81-5d6c60b37492 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:46.437836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8b4d8-d0bf-4369-bb81-5d6c60b37492 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:46.438565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8b4d8-d0bf-4369-bb81-5d6c60b37492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:46.438853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8b4d8-d0bf-4369-bb81-5d6c60b37492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:46.440201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8b4d8-d0bf-4369-bb81-5d6c60b37492 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:46.452447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 635/1270] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:46.458721 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e408df-1586-40bd-8410-860b5fed2508 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:46.493415 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:46.565274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e408df-1586-40bd-8410-860b5fed2508 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:46.600638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66f6c8-d7e6-4aff-afc8-cbf0f7245282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:46.601467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66f6c8-d7e6-4aff-afc8-cbf0f7245282 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:46.601755 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66f6c8-d7e6-4aff-afc8-cbf0f7245282 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:46.602037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66f6c8-d7e6-4aff-afc8-cbf0f7245282 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:46.608910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66f6c8-d7e6-4aff-afc8-cbf0f7245282 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:46.610409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66f6c8-d7e6-4aff-afc8-cbf0f7245282 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-bc66f6c8-d7e6-4aff-afc8-cbf0f7245282 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:25:46.625200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:46.626337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:46.627333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:25:46.627333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:25:46.627333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:25:46.627333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:25:46.627333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:25:46.627333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:25:46.627333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:25:46.627333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:46.627333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 636/1271] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:25:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:46.632384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8b4d8-d0bf-4369-bb81-5d6c60b37492 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:46.694843 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:46.738384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8b4d8-d0bf-4369-bb81-5d6c60b37492 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:46.771922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2345c-0fa6-4624-ab5c-d41fd3b94722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:46.772573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2345c-0fa6-4624-ab5c-d41fd3b94722 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:46.772851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2345c-0fa6-4624-ab5c-d41fd3b94722 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:46.773147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2345c-0fa6-4624-ab5c-d41fd3b94722 None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:46.773925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2345c-0fa6-4624-ab5c-d41fd3b94722 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:46.774229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2345c-0fa6-4624-ab5c-d41fd3b94722 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:46.775635 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2345c-0fa6-4624-ab5c-d41fd3b94722 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:46.787836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 636/1272] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:25:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:46.793846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66f6c8-d7e6-4aff-afc8-cbf0f7245282 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:46.890500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:46.901463 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66f6c8-d7e6-4aff-afc8-cbf0f7245282 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:46.936056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82aee7-879c-4583-b7d6-3da77640c42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:46.936941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82aee7-879c-4583-b7d6-3da77640c42e None admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:46.937255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82aee7-879c-4583-b7d6-3da77640c42e None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:46.937538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82aee7-879c-4583-b7d6-3da77640c42e None admin] PATH_INFO: `/v3/projects/07b861f95bf3498d990785ba72474d82/users/7a7d1e8a0f4b4511a7dc8b27651744c8/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:46.939501 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82aee7-879c-4583-b7d6-3da77640c42e None admin] RBAC: Authorizing `identity:create_grant(project_id=07b861f95bf3498d990785ba72474d82, user_id=7a7d1e8a0f4b4511a7dc8b27651744c8, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:46.941260 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82aee7-879c-4583-b7d6-3da77640c42e None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:46.957518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 637/1273] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jun 7 08:25:46 2022] PUT /identity/v3/projects/07b861f95bf3498d990785ba72474d82/users/7a7d1e8a0f4b4511a7dc8b27651744c8/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:25:49.284012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d701542-873c-4666-a3a8-4c97e63163a5 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:49.284531 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d701542-873c-4666-a3a8-4c97e63163a5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:49.284531 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d701542-873c-4666-a3a8-4c97e63163a5 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:49.286533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 637/1274] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:25:49.291997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a83450-db25-4df5-998c-79c7d6c72400 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:49.292425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a83450-db25-4df5-998c-79c7d6c72400 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:49.292560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a83450-db25-4df5-998c-79c7d6c72400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:49.312610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-00a83450-db25-4df5-998c-79c7d6c72400 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:25:49.313967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-00a83450-db25-4df5-998c-79c7d6c72400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:25:49.390867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 638/1275] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:25:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:25:49.415024 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d701542-873c-4666-a3a8-4c97e63163a5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:49.517703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d701542-873c-4666-a3a8-4c97e63163a5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:49.535135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:49.552584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc1d33-76bc-46f9-98df-baa78fe5631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:49.553529 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc1d33-76bc-46f9-98df-baa78fe5631b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:49.553815 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc1d33-76bc-46f9-98df-baa78fe5631b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:49.554087 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc1d33-76bc-46f9-98df-baa78fe5631b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:49.563918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc1d33-76bc-46f9-98df-baa78fe5631b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-72dc1d33-76bc-46f9-98df-baa78fe5631b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:25:49.572420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:49.573545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:49.574581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:49.574581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:49.574581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:49.574581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:49.574581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:25:49.574581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:25:49.574581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:25:49.574581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:25:49.574581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:25:49.574581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:25:49.574581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:25:49.574581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:49.574581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 638/1276] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jun 7 08:25:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:49.580085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a83450-db25-4df5-998c-79c7d6c72400 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:49.659401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:49.679406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a83450-db25-4df5-998c-79c7d6c72400 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:49.713332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75240fa6-3fcc-41ec-869a-9bd9d428b1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:49.713998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75240fa6-3fcc-41ec-869a-9bd9d428b1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:49.714288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75240fa6-3fcc-41ec-869a-9bd9d428b1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:49.714568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75240fa6-3fcc-41ec-869a-9bd9d428b1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:49.715314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75240fa6-3fcc-41ec-869a-9bd9d428b1f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:49.715624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75240fa6-3fcc-41ec-869a-9bd9d428b1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:49.717000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75240fa6-3fcc-41ec-869a-9bd9d428b1f6 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:49.725845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 639/1277] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jun 7 08:25:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:49.731579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc1d33-76bc-46f9-98df-baa78fe5631b None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:49.834185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc1d33-76bc-46f9-98df-baa78fe5631b None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:49.867473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3badb-bcd1-44bb-a831-2ea5039be6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:49.868267 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3badb-bcd1-44bb-a831-2ea5039be6fa None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:49.868554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3badb-bcd1-44bb-a831-2ea5039be6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:49.868834 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3badb-bcd1-44bb-a831-2ea5039be6fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:49.879391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3badb-bcd1-44bb-a831-2ea5039be6fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:49.880872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3badb-bcd1-44bb-a831-2ea5039be6fa None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-dce3badb-bcd1-44bb-a831-2ea5039be6fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:25:49.898344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:49.899450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:49.900722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:25:49.900722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:25:49.900722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:49.900722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:49.900722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 639/1278] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:49.905346 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75240fa6-3fcc-41ec-869a-9bd9d428b1f6 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:50.003269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75240fa6-3fcc-41ec-869a-9bd9d428b1f6 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:50.037553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff215c-3bd8-4f0f-8eae-79896c9670be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:50.038192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff215c-3bd8-4f0f-8eae-79896c9670be None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:50.038478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff215c-3bd8-4f0f-8eae-79896c9670be None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:50.038753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff215c-3bd8-4f0f-8eae-79896c9670be None admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:50.039469 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff215c-3bd8-4f0f-8eae-79896c9670be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:50.039775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff215c-3bd8-4f0f-8eae-79896c9670be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:50.041156 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff215c-3bd8-4f0f-8eae-79896c9670be None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:50.052747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 640/1279] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:50.058457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3badb-bcd1-44bb-a831-2ea5039be6fa None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:50.089660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:50.157968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3badb-bcd1-44bb-a831-2ea5039be6fa None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:50.191391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb63d2-9f7d-46e2-97c8-658d3507750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:50.192262 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb63d2-9f7d-46e2-97c8-658d3507750e None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:50.192551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb63d2-9f7d-46e2-97c8-658d3507750e None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:50.192920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb63d2-9f7d-46e2-97c8-658d3507750e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:50.200468 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb63d2-9f7d-46e2-97c8-658d3507750e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:50.202009 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb63d2-9f7d-46e2-97c8-658d3507750e None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-4ddb63d2-9f7d-46e2-97c8-658d3507750e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:25:50.217148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:50.218334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:50.219303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:50.219303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:50.219303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:25:50.219303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:25:50.219303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:25:50.219303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:25:50.219303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:25:50.219303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:25:50.219303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:25:50.219303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:50.219303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 640/1280] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:50.224852 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff215c-3bd8-4f0f-8eae-79896c9670be None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:50.228658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:50.325266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff215c-3bd8-4f0f-8eae-79896c9670be None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:50.359888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304ccd6-7f44-4a71-9434-0a8c7d3edea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:50.360475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304ccd6-7f44-4a71-9434-0a8c7d3edea7 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:50.360763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304ccd6-7f44-4a71-9434-0a8c7d3edea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:50.361042 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304ccd6-7f44-4a71-9434-0a8c7d3edea7 None admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:50.361912 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304ccd6-7f44-4a71-9434-0a8c7d3edea7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:50.362221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304ccd6-7f44-4a71-9434-0a8c7d3edea7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:50.363647 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304ccd6-7f44-4a71-9434-0a8c7d3edea7 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:50.381487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 641/1281] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Jun 7 08:25:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:50.387482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb63d2-9f7d-46e2-97c8-658d3507750e None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:50.479216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:50.489390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb63d2-9f7d-46e2-97c8-658d3507750e None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:50.523072 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3f298-f25d-43ce-a502-b966ca9c42d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:50.523716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3f298-f25d-43ce-a502-b966ca9c42d9 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:50.523997 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3f298-f25d-43ce-a502-b966ca9c42d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:50.524272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3f298-f25d-43ce-a502-b966ca9c42d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:50.589276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3f298-f25d-43ce-a502-b966ca9c42d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:50.590723 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3f298-f25d-43ce-a502-b966ca9c42d9 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:50.675237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:50.685680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 641/1282] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jun 7 08:25:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:25:50.691700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304ccd6-7f44-4a71-9434-0a8c7d3edea7 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:50.753148 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:50.791453 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304ccd6-7f44-4a71-9434-0a8c7d3edea7 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:50.825508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a263f-f912-4d82-9c47-cb3881f702e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:50.826298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a263f-f912-4d82-9c47-cb3881f702e6 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:50.826583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a263f-f912-4d82-9c47-cb3881f702e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:50.826860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a263f-f912-4d82-9c47-cb3881f702e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:50.837937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a263f-f912-4d82-9c47-cb3881f702e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:50.839387 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a263f-f912-4d82-9c47-cb3881f702e6 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-668a263f-f912-4d82-9c47-cb3881f702e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:25:50.857623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:50.859153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:50.860270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:25:50.860270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:25:50.860270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:25:50.860270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:50.860270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 642/1283] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:50.864553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3f298-f25d-43ce-a502-b966ca9c42d9 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:50.968845 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3f298-f25d-43ce-a502-b966ca9c42d9 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:51.002827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e03aea-52b9-429c-b7a6-128f2a5dcb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:51.003484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e03aea-52b9-429c-b7a6-128f2a5dcb98 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:51.003762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e03aea-52b9-429c-b7a6-128f2a5dcb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:51.004035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e03aea-52b9-429c-b7a6-128f2a5dcb98 None admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:51.004787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e03aea-52b9-429c-b7a6-128f2a5dcb98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:51.005118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e03aea-52b9-429c-b7a6-128f2a5dcb98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:51.006491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e03aea-52b9-429c-b7a6-128f2a5dcb98 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:51.017816 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 642/1284] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jun 7 08:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:51.024103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a263f-f912-4d82-9c47-cb3881f702e6 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:51.122833 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a263f-f912-4d82-9c47-cb3881f702e6 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:51.137699 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:51.159028 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd92bef-41dc-47ef-a5fd-03f2d23096c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:51.159828 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd92bef-41dc-47ef-a5fd-03f2d23096c8 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:51.160114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd92bef-41dc-47ef-a5fd-03f2d23096c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:51.160392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd92bef-41dc-47ef-a5fd-03f2d23096c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:51.167480 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd92bef-41dc-47ef-a5fd-03f2d23096c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:51.168954 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd92bef-41dc-47ef-a5fd-03f2d23096c8 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application [None req-dcd92bef-41dc-47ef-a5fd-03f2d23096c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:25:51.182840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:51.183908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:51.184884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:25:51.184884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:25:51.184884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:25:51.184884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:25:51.184884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:25:51.184884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:25:51.184884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:25:51.184884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: ERROR keystone.server.flask.application Jun 07 08:25:51.184884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 643/1285] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jun 7 08:25:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:51.189911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e03aea-52b9-429c-b7a6-128f2a5dcb98 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:51.239167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:51.289076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e03aea-52b9-429c-b7a6-128f2a5dcb98 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:51.324647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632ec3d-823d-4c61-8035-75df1657ca3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:51.325343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632ec3d-823d-4c61-8035-75df1657ca3b None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:51.325628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632ec3d-823d-4c61-8035-75df1657ca3b None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:51.325905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632ec3d-823d-4c61-8035-75df1657ca3b None admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:51.326684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632ec3d-823d-4c61-8035-75df1657ca3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:51.326995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632ec3d-823d-4c61-8035-75df1657ca3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:51.328416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632ec3d-823d-4c61-8035-75df1657ca3b None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:51.340645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 643/1286] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jun 7 08:25:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:51.347197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd92bef-41dc-47ef-a5fd-03f2d23096c8 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:51.448564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd92bef-41dc-47ef-a5fd-03f2d23096c8 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:51.483904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506f6db-34ed-4acf-a336-e68616acb6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:51.484559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506f6db-34ed-4acf-a336-e68616acb6db None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:51.484848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506f6db-34ed-4acf-a336-e68616acb6db None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:51.485287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506f6db-34ed-4acf-a336-e68616acb6db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:51.485983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506f6db-34ed-4acf-a336-e68616acb6db None admin] RBAC: Adding query filter params (role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f, scope.project.id=07b861f95bf3498d990785ba72474d82, user.id=7a7d1e8a0f4b4511a7dc8b27651744c8) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:51.486298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506f6db-34ed-4acf-a336-e68616acb6db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:51.487741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506f6db-34ed-4acf-a336-e68616acb6db None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:51.494983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 644/1287] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jun 7 08:25:51 2022] GET /identity/v3/role_assignments?user.id=7a7d1e8a0f4b4511a7dc8b27651744c8&scope.project.id=07b861f95bf3498d990785ba72474d82&role.id=2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:53.757724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1bb02-79d7-42bc-b689-16799acc667f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:53.758292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1bb02-79d7-42bc-b689-16799acc667f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:53.758292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1bb02-79d7-42bc-b689-16799acc667f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:53.759648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 644/1288] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:25:53.765221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316911fa-5461-4377-ab2a-d1e70c22d0b5 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:53.765687 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316911fa-5461-4377-ab2a-d1e70c22d0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:53.765826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316911fa-5461-4377-ab2a-d1e70c22d0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:53.783251 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-316911fa-5461-4377-ab2a-d1e70c22d0b5 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:25:53.784426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-316911fa-5461-4377-ab2a-d1e70c22d0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:25:53.814362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:53.853372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 645/1289] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:25:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:25:53.876184 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1bb02-79d7-42bc-b689-16799acc667f None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:53.982581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1bb02-79d7-42bc-b689-16799acc667f None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:53.993411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:54.016995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcd609-4311-4539-b8a5-be1530e3a862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:54.017736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcd609-4311-4539-b8a5-be1530e3a862 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:54.018016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcd609-4311-4539-b8a5-be1530e3a862 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:54.018287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcd609-4311-4539-b8a5-be1530e3a862 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:54.018921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fcd609-4311-4539-b8a5-be1530e3a862 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:54.021260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fcd609-4311-4539-b8a5-be1530e3a862 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-42fcd609-4311-4539-b8a5-be1530e3a862 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:54.032703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:54.033954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:54.035008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 645/1290] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jun 7 08:25:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:54.041046 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316911fa-5461-4377-ab2a-d1e70c22d0b5 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:54.142606 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316911fa-5461-4377-ab2a-d1e70c22d0b5 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:54.175269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ae37a-0817-4af0-97bd-374aff4412f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:54.175932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ae37a-0817-4af0-97bd-374aff4412f9 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:54.176212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ae37a-0817-4af0-97bd-374aff4412f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:54.176526 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ae37a-0817-4af0-97bd-374aff4412f9 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:54.177450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ae37a-0817-4af0-97bd-374aff4412f9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:54.177756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ae37a-0817-4af0-97bd-374aff4412f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:54.179116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ae37a-0817-4af0-97bd-374aff4412f9 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:54.184151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 646/1291] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Jun 7 08:25:54 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:54.189836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcd609-4311-4539-b8a5-be1530e3a862 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:54.288637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcd609-4311-4539-b8a5-be1530e3a862 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:54.326739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5dd87-1f48-4a2a-8f9a-df4d67e2f500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:54.327491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5dd87-1f48-4a2a-8f9a-df4d67e2f500 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:54.327791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5dd87-1f48-4a2a-8f9a-df4d67e2f500 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:54.328160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5dd87-1f48-4a2a-8f9a-df4d67e2f500 None admin] PATH_INFO: `/v3/services` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:54.328902 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5dd87-1f48-4a2a-8f9a-df4d67e2f500 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:54.329578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5dd87-1f48-4a2a-8f9a-df4d67e2f500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:54.331238 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5dd87-1f48-4a2a-8f9a-df4d67e2f500 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:54.336484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 646/1292] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Jun 7 08:25:54 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:56.651411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee2d10-b54b-4b41-97c0-a907678c35de None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:56.651411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee2d10-b54b-4b41-97c0-a907678c35de None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:56.651411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee2d10-b54b-4b41-97c0-a907678c35de None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:56.651411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 647/1293] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:25:56.656753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a4c83-2c1b-4cdc-976a-1dcc82a80c1a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:56.657272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a4c83-2c1b-4cdc-976a-1dcc82a80c1a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:56.657386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a4c83-2c1b-4cdc-976a-1dcc82a80c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:56.673940 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-8e9a4c83-2c1b-4cdc-976a-1dcc82a80c1a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:25:56.675134 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8e9a4c83-2c1b-4cdc-976a-1dcc82a80c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:25:56.705549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:56.744382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 647/1294] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:25:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:25:56.767800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee2d10-b54b-4b41-97c0-a907678c35de None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:56.798200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:56.869065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee2d10-b54b-4b41-97c0-a907678c35de None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:56.905750 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83669c2-9219-4db1-8283-e3c1098d9369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:56.906374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83669c2-9219-4db1-8283-e3c1098d9369 None admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:56.906657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83669c2-9219-4db1-8283-e3c1098d9369 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:56.906926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83669c2-9219-4db1-8283-e3c1098d9369 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:56.907653 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83669c2-9219-4db1-8283-e3c1098d9369 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:56.909014 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83669c2-9219-4db1-8283-e3c1098d9369 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:56.922694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 648/1295] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jun 7 08:25:56 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:25:59.230135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e34779-255d-49c7-9aef-e8b9362ebe6b None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:59.230645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e34779-255d-49c7-9aef-e8b9362ebe6b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:59.230645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e34779-255d-49c7-9aef-e8b9362ebe6b None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:59.231993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 648/1296] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:25:59.237677 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01884d1c-cda6-45d0-b50f-ce0a6352338a None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:59.238123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01884d1c-cda6-45d0-b50f-ce0a6352338a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:59.238247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01884d1c-cda6-45d0-b50f-ce0a6352338a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:59.255354 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-01884d1c-cda6-45d0-b50f-ce0a6352338a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:25:59.256567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-01884d1c-cda6-45d0-b50f-ce0a6352338a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:25:59.343332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 649/1297] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:25:59 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:25:59.380506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e34779-255d-49c7-9aef-e8b9362ebe6b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:59.490477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e34779-255d-49c7-9aef-e8b9362ebe6b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:59.531661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caadd4-9e25-4354-b3f9-5d9e4095e525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:59.532929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caadd4-9e25-4354-b3f9-5d9e4095e525 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:59.533406 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caadd4-9e25-4354-b3f9-5d9e4095e525 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:59.533852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caadd4-9e25-4354-b3f9-5d9e4095e525 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:59.535065 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caadd4-9e25-4354-b3f9-5d9e4095e525 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:59.537722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caadd4-9e25-4354-b3f9-5d9e4095e525 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-f2caadd4-9e25-4354-b3f9-5d9e4095e525 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:25:59.551545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:25:59.552745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:25:59.553846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 649/1298] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jun 7 08:25:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:25:59.559046 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01884d1c-cda6-45d0-b50f-ce0a6352338a None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:59.577814 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:59.659766 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01884d1c-cda6-45d0-b50f-ce0a6352338a None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:59.695475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a2446-fd23-4f4a-bf89-7ec5590b9d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:59.696138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a2446-fd23-4f4a-bf89-7ec5590b9d73 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:59.696427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a2446-fd23-4f4a-bf89-7ec5590b9d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:59.696707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a2446-fd23-4f4a-bf89-7ec5590b9d73 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:59.697450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a2446-fd23-4f4a-bf89-7ec5590b9d73 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:59.697760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a2446-fd23-4f4a-bf89-7ec5590b9d73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:59.699147 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a2446-fd23-4f4a-bf89-7ec5590b9d73 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:59.704802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 650/1299] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Jun 7 08:25:59 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:59.712021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caadd4-9e25-4354-b3f9-5d9e4095e525 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:59.744122 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:59.814162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caadd4-9e25-4354-b3f9-5d9e4095e525 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:25:59.848482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e116e063-bd00-4933-9d43-a09e771b417c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:25:59.849172 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116e063-bd00-4933-9d43-a09e771b417c None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:25:59.849459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116e063-bd00-4933-9d43-a09e771b417c None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:25:59.849739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116e063-bd00-4933-9d43-a09e771b417c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:25:59.850527 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e116e063-bd00-4933-9d43-a09e771b417c None admin] RBAC: Adding query filter params (interface=public, service_id=d44a33d7416340b6a5980c3bba2c46c7, region_id=RegionOne) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:25:59.850841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e116e063-bd00-4933-9d43-a09e771b417c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:25:59.852192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e116e063-bd00-4933-9d43-a09e771b417c None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:25:59.857358 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 650/1300] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jun 7 08:25:59 2022] GET /identity/v3/endpoints?service_id=d44a33d7416340b6a5980c3bba2c46c7&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:25:59.863088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a2446-fd23-4f4a-bf89-7ec5590b9d73 None admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:25:59.949070 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:25:59.967669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a2446-fd23-4f4a-bf89-7ec5590b9d73 None admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:26:00.003236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9846d22-6267-4602-a767-8d5ccd61f503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:26:00.003824 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9846d22-6267-4602-a767-8d5ccd61f503 None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:26:00.004113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9846d22-6267-4602-a767-8d5ccd61f503 None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:26:00.005104 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9846d22-6267-4602-a767-8d5ccd61f503 None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:26:00.005808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9846d22-6267-4602-a767-8d5ccd61f503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:26:00.007174 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9846d22-6267-4602-a767-8d5ccd61f503 None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:26:00.014120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 651/1301] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jun 7 08:25:59 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:26:02.556390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095df7d5-a6e9-40f4-a387-43574fd185b0 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:26:02.556961 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095df7d5-a6e9-40f4-a387-43574fd185b0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:26:02.556961 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095df7d5-a6e9-40f4-a387-43574fd185b0 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:26:02.558629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 651/1302] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:26:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:26:02.565115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048578b-e05c-4797-9db7-ce453a3153fd None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:26:02.565624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048578b-e05c-4797-9db7-ce453a3153fd None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:26:02.566003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048578b-e05c-4797-9db7-ce453a3153fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:26:02.589681 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-3048578b-e05c-4797-9db7-ce453a3153fd None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:26:02.591442 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-3048578b-e05c-4797-9db7-ce453a3153fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:26:02.668243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 652/1303] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:26:02 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:26:02.692571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095df7d5-a6e9-40f4-a387-43574fd185b0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:26:02.794623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:26:02.802946 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095df7d5-a6e9-40f4-a387-43574fd185b0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:26:02.839065 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4349d-7524-4bff-8a39-611b40234c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:26:02.839756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4349d-7524-4bff-8a39-611b40234c40 None admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:26:02.840032 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4349d-7524-4bff-8a39-611b40234c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:26:02.840309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4349d-7524-4bff-8a39-611b40234c40 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:26:02.840956 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4349d-7524-4bff-8a39-611b40234c40 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:26:02.842356 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4349d-7524-4bff-8a39-611b40234c40 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application [None req-97d4349d-7524-4bff-8a39-611b40234c40 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:26:02.855345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:26:02.857115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application with Lock( Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: ERROR keystone.server.flask.application Jun 07 08:26:02.859161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 652/1304] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jun 7 08:26:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:26:02.862831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048578b-e05c-4797-9db7-ce453a3153fd None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:26:02.955214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:26:02.981403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048578b-e05c-4797-9db7-ce453a3153fd None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:26:03.019259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba0cd9-fbf9-4845-b530-a14da209677d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:26:03.020431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba0cd9-fbf9-4845-b530-a14da209677d None admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:26:03.020710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba0cd9-fbf9-4845-b530-a14da209677d None admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:26:03.020982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba0cd9-fbf9-4845-b530-a14da209677d None admin] PATH_INFO: `/v3/services` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:26:03.021741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba0cd9-fbf9-4845-b530-a14da209677d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:26:03.022163 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba0cd9-fbf9-4845-b530-a14da209677d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:26:03.023543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba0cd9-fbf9-4845-b530-a14da209677d None admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:26:03.028830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 653/1305] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Jun 7 08:26:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:26:03.034646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4349d-7524-4bff-8a39-611b40234c40 None admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:26:03.139981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4349d-7524-4bff-8a39-611b40234c40 None admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:26:03.174380 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47b98b-9676-4c8f-a1db-883a0244d401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:26:03.175053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47b98b-9676-4c8f-a1db-883a0244d401 None admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:26:03.175341 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47b98b-9676-4c8f-a1db-883a0244d401 None admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:26:03.175617 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47b98b-9676-4c8f-a1db-883a0244d401 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:26:03.176398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47b98b-9676-4c8f-a1db-883a0244d401 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:26:03.177819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47b98b-9676-4c8f-a1db-883a0244d401 None admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:26:03.198163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 653/1306] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jun 7 08:26:03 2022] POST /identity/v3/endpoints => generated 331 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:26:13.025706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=30 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:26:13.193232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:09.269610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb072bf7-4d24-442e-80d5-20955d166a56 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:09.270157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb072bf7-4d24-442e-80d5-20955d166a56 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:09.270157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb072bf7-4d24-442e-80d5-20955d166a56 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:09.271614 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 654/1307] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:27:09.276768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a8b4e-ace6-4ca9-80e7-aafcce2c1d3a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:09.277251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a8b4e-ace6-4ca9-80e7-aafcce2c1d3a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:09.277355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a8b4e-ace6-4ca9-80e7-aafcce2c1d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:09.372117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d39a8b4e-ace6-4ca9-80e7-aafcce2c1d3a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:09.373720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d39a8b4e-ace6-4ca9-80e7-aafcce2c1d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:09.548453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 654/1308] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:27:09.607886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781d6b2-e627-41d9-90df-cb259cca4fc1 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:09.608277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781d6b2-e627-41d9-90df-cb259cca4fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:09.608733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781d6b2-e627-41d9-90df-cb259cca4fc1 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:09.610732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 655/1309] 10.222.0.39 () {54 vars in 964 bytes} [Tue Jun 7 08:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:27:09.619115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034202a-8529-4471-b602-769c1d4b648d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:09.619486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034202a-8529-4471-b602-769c1d4b648d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:09.619696 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034202a-8529-4471-b602-769c1d4b648d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:09.760433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9034202a-8529-4471-b602-769c1d4b648d None None] MFA Rules not processed for user `7ddf89deb28749eda2d26ad66bc282f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:09.761688 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9034202a-8529-4471-b602-769c1d4b648d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:09.788404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:09.856300 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 655/1310] 10.222.0.39 () {58 vars in 1061 bytes} [Tue Jun 7 08:27:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:27:09.866102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781d6b2-e627-41d9-90df-cb259cca4fc1 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:27:09.868312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b781d6b2-e627-41d9-90df-cb259cca4fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:10.021706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781d6b2-e627-41d9-90df-cb259cca4fc1 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:27:10.114275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ef6bb-70fc-4846-9923-c83462a945d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:27:10.114930 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ef6bb-70fc-4846-9923-c83462a945d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:10.115220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ef6bb-70fc-4846-9923-c83462a945d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:10.115495 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ef6bb-70fc-4846-9923-c83462a945d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:10.117203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-084ef6bb-70fc-4846-9923-c83462a945d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:10.180687 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:10.245877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ef6bb-70fc-4846-9923-c83462a945d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:27:10.247432 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ef6bb-70fc-4846-9923-c83462a945d2 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:27:10.268791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 656/1311] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:27:12.912121 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6214c7-e244-49f9-9902-ee2518cb3986 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:12.912121 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6214c7-e244-49f9-9902-ee2518cb3986 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:12.912583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6214c7-e244-49f9-9902-ee2518cb3986 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:12.913606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 656/1312] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:27:12.918988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce36470-14e6-4edb-80c6-f957f307b159 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:12.919627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce36470-14e6-4edb-80c6-f957f307b159 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:12.919627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce36470-14e6-4edb-80c6-f957f307b159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:13.002772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-9ce36470-14e6-4edb-80c6-f957f307b159 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:13.003999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9ce36470-14e6-4edb-80c6-f957f307b159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:13.102052 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 657/1313] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:27:13.140192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6214c7-e244-49f9-9902-ee2518cb3986 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:27:13.141773 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ea6214c7-e244-49f9-9902-ee2518cb3986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:13.248811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6214c7-e244-49f9-9902-ee2518cb3986 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:27:13.320617 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:13.323747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d26b95-e020-4b0f-ae2c-1838f92d7ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:27:13.324405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d26b95-e020-4b0f-ae2c-1838f92d7ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:13.324870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d26b95-e020-4b0f-ae2c-1838f92d7ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:13.325179 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d26b95-e020-4b0f-ae2c-1838f92d7ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:13.326820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-29d26b95-e020-4b0f-ae2c-1838f92d7ae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:13.437555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d26b95-e020-4b0f-ae2c-1838f92d7ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:27:13.438998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d26b95-e020-4b0f-ae2c-1838f92d7ae0 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:27:13.458283 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 657/1314] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:27:16.080990 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee6e6b-83f6-49cd-b6d7-85ce068f5b1e None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:16.081552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee6e6b-83f6-49cd-b6d7-85ce068f5b1e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:16.081552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee6e6b-83f6-49cd-b6d7-85ce068f5b1e None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:16.082960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 658/1315] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:27:16.088134 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91648422-335d-4edf-a2f0-49514901a9f8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:16.088568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91648422-335d-4edf-a2f0-49514901a9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:16.088679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91648422-335d-4edf-a2f0-49514901a9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:16.150883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-91648422-335d-4edf-a2f0-49514901a9f8 None None] MFA Rules not processed for user `d49ddf0ef68c45e4a61eddb790691a24`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:16.152018 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-91648422-335d-4edf-a2f0-49514901a9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:16.204407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:16.249599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 658/1316] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:27:16 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:27:16.289981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee6e6b-83f6-49cd-b6d7-85ce068f5b1e None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:27:16.291602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1bee6e6b-83f6-49cd-b6d7-85ce068f5b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:16.301777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:16.397146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee6e6b-83f6-49cd-b6d7-85ce068f5b1e None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:27:16.473780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cba19b-5c18-43ec-8a10-17816d82bd0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:27:16.474348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cba19b-5c18-43ec-8a10-17816d82bd0c service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:16.474636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cba19b-5c18-43ec-8a10-17816d82bd0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:16.474916 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cba19b-5c18-43ec-8a10-17816d82bd0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:16.476565 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-55cba19b-5c18-43ec-8a10-17816d82bd0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:16.538916 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:16.585830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cba19b-5c18-43ec-8a10-17816d82bd0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:27:16.587280 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cba19b-5c18-43ec-8a10-17816d82bd0c service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:27:16.606340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 659/1317] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:27:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:27:19.461571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd444a5-a002-43d3-bb6c-fde3c7bcfb10 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:19.462139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd444a5-a002-43d3-bb6c-fde3c7bcfb10 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:19.462139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd444a5-a002-43d3-bb6c-fde3c7bcfb10 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:19.463491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 659/1318] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:27:19.468812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b671a7-a5be-476b-a810-69c9af48a0ed None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:19.469250 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b671a7-a5be-476b-a810-69c9af48a0ed None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:19.469450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b671a7-a5be-476b-a810-69c9af48a0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:19.535169 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-12b671a7-a5be-476b-a810-69c9af48a0ed None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:19.536359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-12b671a7-a5be-476b-a810-69c9af48a0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:19.761233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 660/1319] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:27:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:27:19.813244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559e482-f464-4208-9c00-675b2a7182af None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:19.813530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559e482-f464-4208-9c00-675b2a7182af None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:19.813809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559e482-f464-4208-9c00-675b2a7182af None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:19.815602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 660/1320] 10.222.0.39 () {54 vars in 964 bytes} [Tue Jun 7 08:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:27:19.821476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b0b3d-7b93-4f75-8565-a79deabf4017 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:19.821770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b0b3d-7b93-4f75-8565-a79deabf4017 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:19.822051 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b0b3d-7b93-4f75-8565-a79deabf4017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:19.842503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:19.898222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-a98b0b3d-7b93-4f75-8565-a79deabf4017 None None] MFA Rules not processed for user `7ddf89deb28749eda2d26ad66bc282f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:19.899461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a98b0b3d-7b93-4f75-8565-a79deabf4017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:19.978307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 661/1321] 10.222.0.39 () {58 vars in 1061 bytes} [Tue Jun 7 08:27:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:27:19.987337 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559e482-f464-4208-9c00-675b2a7182af None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:27:19.988851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4559e482-f464-4208-9c00-675b2a7182af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:20.088445 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559e482-f464-4208-9c00-675b2a7182af None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:27:20.159438 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b899ee-2a50-4082-8d13-24fd8c87a1e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:27:20.160081 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b899ee-2a50-4082-8d13-24fd8c87a1e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:20.160363 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b899ee-2a50-4082-8d13-24fd8c87a1e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:20.160638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b899ee-2a50-4082-8d13-24fd8c87a1e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:20.162290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d8b899ee-2a50-4082-8d13-24fd8c87a1e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:20.169515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:20.268893 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b899ee-2a50-4082-8d13-24fd8c87a1e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:27:20.270357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b899ee-2a50-4082-8d13-24fd8c87a1e4 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:27:20.288735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 661/1322] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:27:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:27:23.791776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29fda8d-18c4-4e87-bd3e-d4bc83b86a3a None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:23.792349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29fda8d-18c4-4e87-bd3e-d4bc83b86a3a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:23.792349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29fda8d-18c4-4e87-bd3e-d4bc83b86a3a None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:23.793895 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 662/1323] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:27:23.799454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381e235-6599-42bd-be89-0ac91a47999a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:23.799892 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381e235-6599-42bd-be89-0ac91a47999a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:23.800010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381e235-6599-42bd-be89-0ac91a47999a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:23.868762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b381e235-6599-42bd-be89-0ac91a47999a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:23.869987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b381e235-6599-42bd-be89-0ac91a47999a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:23.931532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:23.972789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 662/1324] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:27:24.010922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29fda8d-18c4-4e87-bd3e-d4bc83b86a3a None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:27:24.012483 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e29fda8d-18c4-4e87-bd3e-d4bc83b86a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:24.129068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29fda8d-18c4-4e87-bd3e-d4bc83b86a3a None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:27:24.142097 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:24.209261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1369723-472c-4739-b62f-af2c43cf8787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:27:24.209911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1369723-472c-4739-b62f-af2c43cf8787 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:24.210190 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1369723-472c-4739-b62f-af2c43cf8787 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:24.210460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1369723-472c-4739-b62f-af2c43cf8787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:24.212130 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c1369723-472c-4739-b62f-af2c43cf8787 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:24.337442 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1369723-472c-4739-b62f-af2c43cf8787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:27:24.338900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1369723-472c-4739-b62f-af2c43cf8787 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:27:24.360692 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 663/1325] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:27:26.891488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f340e59-7985-4cb0-8652-dfecf723b01a None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:26.892093 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f340e59-7985-4cb0-8652-dfecf723b01a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:26.892093 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f340e59-7985-4cb0-8652-dfecf723b01a None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:26.895170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 663/1326] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:27:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:27:26.900908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165714b2-412f-47c2-8f7d-f014c588b2ad None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:26.901357 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165714b2-412f-47c2-8f7d-f014c588b2ad None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:26.901509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165714b2-412f-47c2-8f7d-f014c588b2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:26.948478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:26.968141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-165714b2-412f-47c2-8f7d-f014c588b2ad None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:26.969454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-165714b2-412f-47c2-8f7d-f014c588b2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:27.072898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 664/1327] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:27:27.110377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f340e59-7985-4cb0-8652-dfecf723b01a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:27:27.112020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5f340e59-7985-4cb0-8652-dfecf723b01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:27.217741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f340e59-7985-4cb0-8652-dfecf723b01a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:27:27.290443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03df269-48c6-45a6-9290-56062eba4a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:27:27.291094 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03df269-48c6-45a6-9290-56062eba4a24 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:27.291377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03df269-48c6-45a6-9290-56062eba4a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:27.291651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03df269-48c6-45a6-9290-56062eba4a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:27.293311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c03df269-48c6-45a6-9290-56062eba4a24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:27.310937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:27.404442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03df269-48c6-45a6-9290-56062eba4a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:27:27.405923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03df269-48c6-45a6-9290-56062eba4a24 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:27:27.423851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 664/1328] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:27:29.857894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96294441-3d7b-4f28-8301-4ee1c9a0096e None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:29.858713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96294441-3d7b-4f28-8301-4ee1c9a0096e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:29.858713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96294441-3d7b-4f28-8301-4ee1c9a0096e None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:29.859909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 665/1329] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:27:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:27:29.866243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a0252-6640-44cb-84c5-0fc26eecdc08 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:29.866760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a0252-6640-44cb-84c5-0fc26eecdc08 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:29.866853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a0252-6640-44cb-84c5-0fc26eecdc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:29.947292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e89a0252-6640-44cb-84c5-0fc26eecdc08 None None] MFA Rules not processed for user `d49ddf0ef68c45e4a61eddb790691a24`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:29.948654 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e89a0252-6640-44cb-84c5-0fc26eecdc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:30.022441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:30.060127 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 665/1330] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:27:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:27:30.105429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96294441-3d7b-4f28-8301-4ee1c9a0096e None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:27:30.107027 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-96294441-3d7b-4f28-8301-4ee1c9a0096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:30.217959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96294441-3d7b-4f28-8301-4ee1c9a0096e None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:27:30.238007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:30.293197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7fa4c-6850-4d7b-ab09-f8ceb96fc282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:27:30.293847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7fa4c-6850-4d7b-ab09-f8ceb96fc282 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:30.294132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7fa4c-6850-4d7b-ab09-f8ceb96fc282 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:30.294406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7fa4c-6850-4d7b-ab09-f8ceb96fc282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:30.296026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e9e7fa4c-6850-4d7b-ab09-f8ceb96fc282 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:30.413199 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7fa4c-6850-4d7b-ab09-f8ceb96fc282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:27:30.414657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7fa4c-6850-4d7b-ab09-f8ceb96fc282 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:27:30.433454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 666/1331] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:27:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:27:32.957810 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f92b78-1a4e-4939-82b6-1d19cca99e82 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:32.958335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f92b78-1a4e-4939-82b6-1d19cca99e82 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:32.958452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f92b78-1a4e-4939-82b6-1d19cca99e82 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:32.959735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 666/1332] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:27:32.965440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28bd2-5c21-45e8-bf88-7b468a9b1aad None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:32.965837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28bd2-5c21-45e8-bf88-7b468a9b1aad None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:32.965996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28bd2-5c21-45e8-bf88-7b468a9b1aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:33.027451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:33.029975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d3b28bd2-5c21-45e8-bf88-7b468a9b1aad None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:33.031819 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d3b28bd2-5c21-45e8-bf88-7b468a9b1aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:33.128999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 667/1333] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:27:33.164583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f92b78-1a4e-4939-82b6-1d19cca99e82 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:27:33.166194 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-37f92b78-1a4e-4939-82b6-1d19cca99e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:33.281604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f92b78-1a4e-4939-82b6-1d19cca99e82 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:27:33.355849 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea976e-4178-4242-9947-0d65c9cb18c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:27:33.356494 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea976e-4178-4242-9947-0d65c9cb18c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:33.356775 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea976e-4178-4242-9947-0d65c9cb18c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:33.357048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea976e-4178-4242-9947-0d65c9cb18c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:33.358696 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-27ea976e-4178-4242-9947-0d65c9cb18c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:33.385589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:33.470439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea976e-4178-4242-9947-0d65c9cb18c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:27:33.471931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea976e-4178-4242-9947-0d65c9cb18c4 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:27:33.491486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 667/1334] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:27:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:27:35.866760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77feb4c-7ed5-4088-b655-b0b625fe9fec None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:35.867276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77feb4c-7ed5-4088-b655-b0b625fe9fec None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:35.867390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77feb4c-7ed5-4088-b655-b0b625fe9fec None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:35.868707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 668/1335] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:27:35.874073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccd85a-f66a-4b69-bbbc-88daed564813 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:35.874501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccd85a-f66a-4b69-bbbc-88daed564813 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:35.874630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccd85a-f66a-4b69-bbbc-88daed564813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:35.946038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-59ccd85a-f66a-4b69-bbbc-88daed564813 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:35.947253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-59ccd85a-f66a-4b69-bbbc-88daed564813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:36.027754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:36.054407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 668/1336] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:27:36.096366 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77feb4c-7ed5-4088-b655-b0b625fe9fec None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:27:36.097938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b77feb4c-7ed5-4088-b655-b0b625fe9fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:36.205558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77feb4c-7ed5-4088-b655-b0b625fe9fec None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:27:36.233988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:36.284334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558e4f5-50a5-4923-847a-e2e7a0123277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:27:36.285121 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558e4f5-50a5-4923-847a-e2e7a0123277 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:36.285304 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558e4f5-50a5-4923-847a-e2e7a0123277 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:36.285581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558e4f5-50a5-4923-847a-e2e7a0123277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:36.287263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-3558e4f5-50a5-4923-847a-e2e7a0123277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:36.402178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558e4f5-50a5-4923-847a-e2e7a0123277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:27:36.403644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558e4f5-50a5-4923-847a-e2e7a0123277 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:27:36.422880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 669/1337] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:27:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:27:39.139235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba033e3-774f-4514-be87-c22453c6d0f1 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:39.139827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba033e3-774f-4514-be87-c22453c6d0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:39.139827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba033e3-774f-4514-be87-c22453c6d0f1 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:39.141210 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 669/1338] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:27:39.147548 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19691fc7-449a-46b9-a637-f1b7b1603515 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:39.147992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19691fc7-449a-46b9-a637-f1b7b1603515 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:39.148099 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19691fc7-449a-46b9-a637-f1b7b1603515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:39.213449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-19691fc7-449a-46b9-a637-f1b7b1603515 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:39.214658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-19691fc7-449a-46b9-a637-f1b7b1603515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:39.223129 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:39.313260 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 670/1339] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:27:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:27:39.351983 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba033e3-774f-4514-be87-c22453c6d0f1 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:27:39.355048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fba033e3-774f-4514-be87-c22453c6d0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:39.474226 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba033e3-774f-4514-be87-c22453c6d0f1 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:27:39.486991 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386a91b-3db7-4063-a528-12b5da7b82fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:27:39.487643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386a91b-3db7-4063-a528-12b5da7b82fc service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:39.487928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386a91b-3db7-4063-a528-12b5da7b82fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:39.488207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386a91b-3db7-4063-a528-12b5da7b82fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:39.489869 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d386a91b-3db7-4063-a528-12b5da7b82fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:39.589880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:39.593435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386a91b-3db7-4063-a528-12b5da7b82fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:27:39.595045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386a91b-3db7-4063-a528-12b5da7b82fc service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:27:39.616175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 670/1340] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:27:39 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:27:43.950147 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004d96f-bd24-41c6-8e05-f9c10c04f181 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:43.950684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004d96f-bd24-41c6-8e05-f9c10c04f181 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:43.950684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004d96f-bd24-41c6-8e05-f9c10c04f181 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:43.952554 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 671/1341] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:27:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:27:43.959192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937c5c9-d372-41d6-9868-68e0abc064f1 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:43.959636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937c5c9-d372-41d6-9868-68e0abc064f1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:43.959759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937c5c9-d372-41d6-9868-68e0abc064f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:44.007152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b937c5c9-d372-41d6-9868-68e0abc064f1 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:44.008306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b937c5c9-d372-41d6-9868-68e0abc064f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:44.088369 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 671/1342] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:27:44.127547 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004d96f-bd24-41c6-8e05-f9c10c04f181 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:27:44.129892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f004d96f-bd24-41c6-8e05-f9c10c04f181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:44.158050 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004d96f-bd24-41c6-8e05-f9c10c04f181 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:27:44.161647 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fdd32-f357-48a9-a881-153d529e15ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:27:44.162294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fdd32-f357-48a9-a881-153d529e15ea service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:44.162601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fdd32-f357-48a9-a881-153d529e15ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:44.162881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fdd32-f357-48a9-a881-153d529e15ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:44.173076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fdd32-f357-48a9-a881-153d529e15ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:27:44.174559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fdd32-f357-48a9-a881-153d529e15ea service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:27:44.184678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 672/1343] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:27:47.077488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f44178-d0a7-442a-b042-1ca107bff6fb None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:47.078012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f44178-d0a7-442a-b042-1ca107bff6fb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:47.078093 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f44178-d0a7-442a-b042-1ca107bff6fb None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:47.080441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 672/1344] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:27:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:27:47.085825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ac3ac-82dc-45c1-9278-c71d7b711d63 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:47.086270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ac3ac-82dc-45c1-9278-c71d7b711d63 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:47.086375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ac3ac-82dc-45c1-9278-c71d7b711d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:47.117651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-ab9ac3ac-82dc-45c1-9278-c71d7b711d63 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:47.118862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ab9ac3ac-82dc-45c1-9278-c71d7b711d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:47.127786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 673/1345] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:27:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:27:47.164076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f44178-d0a7-442a-b042-1ca107bff6fb None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:27:47.176520 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f44178-d0a7-442a-b042-1ca107bff6fb None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:27:47.180501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f7812-b823-4748-8fe8-46441f6329a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:27:47.181223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f7812-b823-4748-8fe8-46441f6329a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:47.181577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f7812-b823-4748-8fe8-46441f6329a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:47.181979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f7812-b823-4748-8fe8-46441f6329a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:47.193172 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f7812-b823-4748-8fe8-46441f6329a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:27:47.194800 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f7812-b823-4748-8fe8-46441f6329a2 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:27:47.208085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 673/1346] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:27:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:27:51.554162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37df985-433c-413d-9684-e5e0e7ab5e07 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:51.554680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37df985-433c-413d-9684-e5e0e7ab5e07 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:51.554772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37df985-433c-413d-9684-e5e0e7ab5e07 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:51.556111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 674/1347] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:27:51.561304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3820111-a0a6-4d2c-b23b-101d8bb070a3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:51.561733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3820111-a0a6-4d2c-b23b-101d8bb070a3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:51.561859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3820111-a0a6-4d2c-b23b-101d8bb070a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:51.580853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c3820111-a0a6-4d2c-b23b-101d8bb070a3 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:51.582099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c3820111-a0a6-4d2c-b23b-101d8bb070a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:51.590007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 674/1348] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:27:51.627594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37df985-433c-413d-9684-e5e0e7ab5e07 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:27:51.642856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37df985-433c-413d-9684-e5e0e7ab5e07 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:27:51.647094 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b147b-27da-4204-9f60-8fe3f2ecde92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:27:51.647862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b147b-27da-4204-9f60-8fe3f2ecde92 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:51.648213 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b147b-27da-4204-9f60-8fe3f2ecde92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:51.648558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b147b-27da-4204-9f60-8fe3f2ecde92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:51.660032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b147b-27da-4204-9f60-8fe3f2ecde92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:27:51.661598 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b147b-27da-4204-9f60-8fe3f2ecde92 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:27:51.672118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 675/1349] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:27:57.222771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab75d4d-1e90-40fa-b2f8-850d23748c08 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:57.223284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab75d4d-1e90-40fa-b2f8-850d23748c08 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:57.223367 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab75d4d-1e90-40fa-b2f8-850d23748c08 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:57.224673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 675/1350] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jun 7 08:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:27:57.230082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b08b7-41de-494e-80ed-8d657e934d5f None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:27:57.231005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b08b7-41de-494e-80ed-8d657e934d5f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:27:57.231472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b08b7-41de-494e-80ed-8d657e934d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:27:57.319938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-bd1b08b7-41de-494e-80ed-8d657e934d5f None None] MFA Rules not processed for user `7a7d1e8a0f4b4511a7dc8b27651744c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:27:57.321439 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-bd1b08b7-41de-494e-80ed-8d657e934d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:27:57.366738 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:27:57.414144 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 676/1351] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jun 7 08:27:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:01.574552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=60 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:28:01.806912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77f2fe-b020-4078-ab78-a2155d6cc341 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:01.807277 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77f2fe-b020-4078-ab78-a2155d6cc341 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:01.807478 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77f2fe-b020-4078-ab78-a2155d6cc341 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:01.809115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 676/1352] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jun 7 08:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:01.814855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963717d-32f2-4c69-9b27-9f6678f6dc4e None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:01.815605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963717d-32f2-4c69-9b27-9f6678f6dc4e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:01.815605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963717d-32f2-4c69-9b27-9f6678f6dc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:01.835149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1963717d-32f2-4c69-9b27-9f6678f6dc4e None None] MFA Rules not processed for user `7a7d1e8a0f4b4511a7dc8b27651744c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:01.836255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1963717d-32f2-4c69-9b27-9f6678f6dc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:01.845400 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 677/1353] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jun 7 08:28:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:03.532408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42440c5b-4c8e-4730-bd28-588d6141578a None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:03.533044 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42440c5b-4c8e-4730-bd28-588d6141578a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:03.533044 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42440c5b-4c8e-4730-bd28-588d6141578a None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:03.534291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 677/1354] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jun 7 08:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:03.539568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc38e19-af2e-48ee-8ec3-e49390c81b32 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:03.540088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc38e19-af2e-48ee-8ec3-e49390c81b32 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:03.540199 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc38e19-af2e-48ee-8ec3-e49390c81b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:03.560853 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-bdc38e19-af2e-48ee-8ec3-e49390c81b32 None None] MFA Rules not processed for user `7a7d1e8a0f4b4511a7dc8b27651744c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:03.562216 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-bdc38e19-af2e-48ee-8ec3-e49390c81b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:03.571896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 678/1355] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jun 7 08:28:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:07.463446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4629b-fe25-46be-b244-0f44d6ac9fd2 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:07.463976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4629b-fe25-46be-b244-0f44d6ac9fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:07.463976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4629b-fe25-46be-b244-0f44d6ac9fd2 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:07.466296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 678/1356] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:07.472127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e24dd-2d7a-45a3-833a-79d4540d1d74 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:07.472674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e24dd-2d7a-45a3-833a-79d4540d1d74 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:07.472674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e24dd-2d7a-45a3-833a-79d4540d1d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:07.491836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-7d9e24dd-2d7a-45a3-833a-79d4540d1d74 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:07.493209 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-7d9e24dd-2d7a-45a3-833a-79d4540d1d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:07.502459 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 679/1357] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:07.565625 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378ac49-46d9-41e0-845f-c16c4e50d801 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:07.566142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378ac49-46d9-41e0-845f-c16c4e50d801 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:07.566739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378ac49-46d9-41e0-845f-c16c4e50d801 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:07.569014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 679/1358] 10.222.0.39 () {54 vars in 961 bytes} [Tue Jun 7 08:28:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:07.583137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96837b-b5e9-46ab-a3a1-826cd52e65f9 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:07.583605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96837b-b5e9-46ab-a3a1-826cd52e65f9 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:07.584034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96837b-b5e9-46ab-a3a1-826cd52e65f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:07.645860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-0e96837b-b5e9-46ab-a3a1-826cd52e65f9 None None] MFA Rules not processed for user `3f2a2bf913d1412f996acf4401ba7673`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:07.647105 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0e96837b-b5e9-46ab-a3a1-826cd52e65f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:07.761470 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 680/1359] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Jun 7 08:28:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:07.770948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378ac49-46d9-41e0-845f-c16c4e50d801 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:07.788254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378ac49-46d9-41e0-845f-c16c4e50d801 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:07.794439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404c845-6a59-447d-96cd-9bbdaffceb1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:07.795448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404c845-6a59-447d-96cd-9bbdaffceb1d service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:07.795969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404c845-6a59-447d-96cd-9bbdaffceb1d service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:07.796450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404c845-6a59-447d-96cd-9bbdaffceb1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:07.807170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404c845-6a59-447d-96cd-9bbdaffceb1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:07.809338 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404c845-6a59-447d-96cd-9bbdaffceb1d service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:07.824249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 680/1360] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:28:10.429981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304b90-e033-485f-9c6d-7296f76395ee None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:10.430589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304b90-e033-485f-9c6d-7296f76395ee None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:10.430589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304b90-e033-485f-9c6d-7296f76395ee None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:10.432103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 681/1361] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:10.439058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654f230-7293-41e7-a602-d92dbcaa2af8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:10.439553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654f230-7293-41e7-a602-d92dbcaa2af8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:10.439693 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654f230-7293-41e7-a602-d92dbcaa2af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:10.458752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0654f230-7293-41e7-a602-d92dbcaa2af8 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:10.460082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0654f230-7293-41e7-a602-d92dbcaa2af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:10.468537 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 681/1362] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:10.515225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304b90-e033-485f-9c6d-7296f76395ee None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:10.533722 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304b90-e033-485f-9c6d-7296f76395ee None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:10.539441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba88b-d843-497a-b83c-34f19e40ec6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:10.540088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba88b-d843-497a-b83c-34f19e40ec6b service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:10.540369 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba88b-d843-497a-b83c-34f19e40ec6b service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:10.540643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba88b-d843-497a-b83c-34f19e40ec6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:10.549818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ba88b-d843-497a-b83c-34f19e40ec6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:10.551402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ba88b-d843-497a-b83c-34f19e40ec6b service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:10.563487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 682/1363] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:28:13.224635 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d8444-51d8-4e68-8a3d-9c7c18b3396e None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:13.225167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d8444-51d8-4e68-8a3d-9c7c18b3396e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:13.225293 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d8444-51d8-4e68-8a3d-9c7c18b3396e None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:13.226610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 682/1364] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:13.231974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7ca2-3200-445e-8f15-551399a3626c None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:13.232542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7ca2-3200-445e-8f15-551399a3626c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:13.232542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7ca2-3200-445e-8f15-551399a3626c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:13.251098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-ce1c7ca2-3200-445e-8f15-551399a3626c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:13.252498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ce1c7ca2-3200-445e-8f15-551399a3626c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:13.262063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 683/1365] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:13.322039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94321ed7-7841-4194-a89a-7d3c0546e756 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:13.322434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94321ed7-7841-4194-a89a-7d3c0546e756 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:13.322716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94321ed7-7841-4194-a89a-7d3c0546e756 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:13.324068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 683/1366] 10.222.0.39 () {54 vars in 961 bytes} [Tue Jun 7 08:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:13.331030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a245ec9-4825-4f39-929a-da91c4a01dde None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:13.331345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a245ec9-4825-4f39-929a-da91c4a01dde None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:13.331624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a245ec9-4825-4f39-929a-da91c4a01dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:13.349931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1a245ec9-4825-4f39-929a-da91c4a01dde None None] MFA Rules not processed for user `3f2a2bf913d1412f996acf4401ba7673`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:13.351151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1a245ec9-4825-4f39-929a-da91c4a01dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:13.361181 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 684/1367] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Jun 7 08:28:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:28:13.368907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94321ed7-7841-4194-a89a-7d3c0546e756 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:13.384058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94321ed7-7841-4194-a89a-7d3c0546e756 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:13.389450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344e444-39e4-4da6-ac62-65cc456a7bb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:13.390083 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344e444-39e4-4da6-ac62-65cc456a7bb6 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:13.390366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344e444-39e4-4da6-ac62-65cc456a7bb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:13.390679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344e444-39e4-4da6-ac62-65cc456a7bb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:13.399244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344e444-39e4-4da6-ac62-65cc456a7bb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:13.400808 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344e444-39e4-4da6-ac62-65cc456a7bb6 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:13.410651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 684/1368] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:28:15.775018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a974-3fff-437d-b661-79d052f9444f None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:15.775747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a974-3fff-437d-b661-79d052f9444f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:15.776135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a974-3fff-437d-b661-79d052f9444f None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:15.778204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 685/1369] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:15.784591 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7ef46-af2b-40ed-b173-f2056fdfb411 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:15.785440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7ef46-af2b-40ed-b173-f2056fdfb411 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:15.785893 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7ef46-af2b-40ed-b173-f2056fdfb411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:15.805330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c1d7ef46-af2b-40ed-b173-f2056fdfb411 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:15.806820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c1d7ef46-af2b-40ed-b173-f2056fdfb411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:15.816106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 685/1370] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:15.862173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59a974-3fff-437d-b661-79d052f9444f None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:15.879821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59a974-3fff-437d-b661-79d052f9444f None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:15.885670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac6cfa-761d-4bd3-a356-252c44555efa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:15.886502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac6cfa-761d-4bd3-a356-252c44555efa service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:15.886926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac6cfa-761d-4bd3-a356-252c44555efa service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:15.887337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac6cfa-761d-4bd3-a356-252c44555efa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:15.898421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac6cfa-761d-4bd3-a356-252c44555efa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:15.900217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac6cfa-761d-4bd3-a356-252c44555efa service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:15.913338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 686/1371] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:28:16.095319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716fd1a-0d78-40fc-a9c4-15524021e43c None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:16.097497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716fd1a-0d78-40fc-a9c4-15524021e43c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:16.098063 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716fd1a-0d78-40fc-a9c4-15524021e43c None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:16.100203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 686/1372] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:16.106067 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e507786-79b8-4912-8e86-5b98b448651c None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:16.108725 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e507786-79b8-4912-8e86-5b98b448651c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:16.109329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e507786-79b8-4912-8e86-5b98b448651c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:16.134892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-2e507786-79b8-4912-8e86-5b98b448651c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:16.136736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-2e507786-79b8-4912-8e86-5b98b448651c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:16.148523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 687/1373] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:28:16.490796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba596eb-684d-4144-98c6-c090e429d58e None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:16.491204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba596eb-684d-4144-98c6-c090e429d58e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:16.491381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba596eb-684d-4144-98c6-c090e429d58e None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:16.492722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 687/1374] 10.222.0.39 () {54 vars in 963 bytes} [Tue Jun 7 08:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:16.498475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1181c7-a2c8-448e-929e-dd0e1bd0f955 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:16.498785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1181c7-a2c8-448e-929e-dd0e1bd0f955 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:16.499047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1181c7-a2c8-448e-929e-dd0e1bd0f955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:16.530495 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:28:16.560795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-9e1181c7-a2c8-448e-929e-dd0e1bd0f955 None None] MFA Rules not processed for user `2b6943045d594d729fc7c4b15cbaa07e`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:16.562409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9e1181c7-a2c8-448e-929e-dd0e1bd0f955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:16.640435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 688/1375] 10.222.0.39 () {58 vars in 1060 bytes} [Tue Jun 7 08:28:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:28:16.647827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba596eb-684d-4144-98c6-c090e429d58e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:16.659592 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba596eb-684d-4144-98c6-c090e429d58e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:16.663299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab597f-992f-4f98-a867-86735d37b282 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:16.663953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab597f-992f-4f98-a867-86735d37b282 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:16.664236 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab597f-992f-4f98-a867-86735d37b282 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:16.664515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab597f-992f-4f98-a867-86735d37b282 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:16.675652 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab597f-992f-4f98-a867-86735d37b282 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:16.677737 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab597f-992f-4f98-a867-86735d37b282 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:16.689873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 688/1376] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:28:18.345882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228c941-ef3d-484f-b268-990566b2dc89 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:18.346425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228c941-ef3d-484f-b268-990566b2dc89 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:18.346425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228c941-ef3d-484f-b268-990566b2dc89 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:18.347826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 689/1377] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:18.353354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a615dfd-6487-4c8c-a300-26aacc0fb6b3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:18.353799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a615dfd-6487-4c8c-a300-26aacc0fb6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:18.353799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a615dfd-6487-4c8c-a300-26aacc0fb6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:18.371542 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3a615dfd-6487-4c8c-a300-26aacc0fb6b3 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:18.372747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3a615dfd-6487-4c8c-a300-26aacc0fb6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:18.382215 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 689/1378] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:28:18.428874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228c941-ef3d-484f-b268-990566b2dc89 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:18.444517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228c941-ef3d-484f-b268-990566b2dc89 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:18.450158 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77057cb2-c954-4720-adaf-8d4ff181c2d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:18.450903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77057cb2-c954-4720-adaf-8d4ff181c2d9 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:18.451348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77057cb2-c954-4720-adaf-8d4ff181c2d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:18.451633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77057cb2-c954-4720-adaf-8d4ff181c2d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:18.462435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77057cb2-c954-4720-adaf-8d4ff181c2d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:18.464015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77057cb2-c954-4720-adaf-8d4ff181c2d9 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:18.475523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 690/1379] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:28:20.833612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109e289-4861-488a-aa85-3ef0e8f72152 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:20.834144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109e289-4861-488a-aa85-3ef0e8f72152 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:20.834144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109e289-4861-488a-aa85-3ef0e8f72152 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:20.835546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 690/1380] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:20.840895 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ea3ed-fd23-4845-9d35-63e8774f7f40 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:20.841340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ea3ed-fd23-4845-9d35-63e8774f7f40 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:20.841506 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ea3ed-fd23-4845-9d35-63e8774f7f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:20.860759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-940ea3ed-fd23-4845-9d35-63e8774f7f40 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:20.862277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-940ea3ed-fd23-4845-9d35-63e8774f7f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:20.870347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 691/1381] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:20.916169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109e289-4861-488a-aa85-3ef0e8f72152 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:20.932093 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109e289-4861-488a-aa85-3ef0e8f72152 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:20.937796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615c72b-43bf-4740-8903-f3af39801b07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:20.938539 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615c72b-43bf-4740-8903-f3af39801b07 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:20.938831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615c72b-43bf-4740-8903-f3af39801b07 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:20.939116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615c72b-43bf-4740-8903-f3af39801b07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:20.948175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615c72b-43bf-4740-8903-f3af39801b07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:20.949793 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615c72b-43bf-4740-8903-f3af39801b07 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:20.961200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 691/1382] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:28:23.285468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002edaf-53c1-4dc8-820d-3226c9ba1b1a None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:23.286046 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002edaf-53c1-4dc8-820d-3226c9ba1b1a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:23.286265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002edaf-53c1-4dc8-820d-3226c9ba1b1a None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:23.287755 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 692/1383] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:23.293193 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028af25-0470-475c-8c0f-de4c83f30ffe None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:23.293653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028af25-0470-475c-8c0f-de4c83f30ffe None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:23.293740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028af25-0470-475c-8c0f-de4c83f30ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:23.314161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d028af25-0470-475c-8c0f-de4c83f30ffe None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:23.315592 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d028af25-0470-475c-8c0f-de4c83f30ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:23.324200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 692/1384] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:23.370375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002edaf-53c1-4dc8-820d-3226c9ba1b1a None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:23.386030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002edaf-53c1-4dc8-820d-3226c9ba1b1a None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:23.392042 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0851c-21bd-4bf3-9098-e08396dd54e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:23.392710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0851c-21bd-4bf3-9098-e08396dd54e3 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:23.392999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0851c-21bd-4bf3-9098-e08396dd54e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:23.393312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0851c-21bd-4bf3-9098-e08396dd54e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:23.402930 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0851c-21bd-4bf3-9098-e08396dd54e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:23.404541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0851c-21bd-4bf3-9098-e08396dd54e3 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:23.418514 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 693/1385] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:28:26.024411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018a1dc-626c-4f5a-a808-7d75251bce9a None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:26.024975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018a1dc-626c-4f5a-a808-7d75251bce9a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:26.024975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018a1dc-626c-4f5a-a808-7d75251bce9a None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:26.026611 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 693/1386] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:26.032361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bff5d4-3382-45c4-a331-3a17510a297f None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:26.032808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bff5d4-3382-45c4-a331-3a17510a297f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:26.033062 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bff5d4-3382-45c4-a331-3a17510a297f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:26.051310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-c1bff5d4-3382-45c4-a331-3a17510a297f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:26.052510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c1bff5d4-3382-45c4-a331-3a17510a297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:26.060561 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 694/1387] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:26.105660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018a1dc-626c-4f5a-a808-7d75251bce9a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:26.119965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018a1dc-626c-4f5a-a808-7d75251bce9a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:26.125377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f252d-c036-4595-9442-a87159586886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:26.126017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f252d-c036-4595-9442-a87159586886 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:26.126329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f252d-c036-4595-9442-a87159586886 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:26.126607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f252d-c036-4595-9442-a87159586886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:26.135929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f252d-c036-4595-9442-a87159586886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:26.137804 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f252d-c036-4595-9442-a87159586886 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:26.148240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 694/1388] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:28:28.477027 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dd3ed-fe7f-4ebf-9bdf-fe8fe3732239 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:28.477618 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dd3ed-fe7f-4ebf-9bdf-fe8fe3732239 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:28.477820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dd3ed-fe7f-4ebf-9bdf-fe8fe3732239 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:28.480184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 695/1389] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:28:28.486453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc05fc-e8d6-49e0-a6b5-207cdef3557a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:28.487103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc05fc-e8d6-49e0-a6b5-207cdef3557a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:28.487103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc05fc-e8d6-49e0-a6b5-207cdef3557a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:28.507888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c2cc05fc-e8d6-49e0-a6b5-207cdef3557a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:28.509213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c2cc05fc-e8d6-49e0-a6b5-207cdef3557a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:28.518975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 695/1390] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:28.566081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9dd3ed-fe7f-4ebf-9bdf-fe8fe3732239 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:28.582898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9dd3ed-fe7f-4ebf-9bdf-fe8fe3732239 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:28.588778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8e679-1132-4eca-b36c-2ffa6cfce9e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:28.589449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8e679-1132-4eca-b36c-2ffa6cfce9e2 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:28.589762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8e679-1132-4eca-b36c-2ffa6cfce9e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:28.590045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8e679-1132-4eca-b36c-2ffa6cfce9e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:28.600979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8e679-1132-4eca-b36c-2ffa6cfce9e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:28.602781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8e679-1132-4eca-b36c-2ffa6cfce9e2 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:28.614115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 696/1391] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:28:30.983853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5a541-1515-4e97-8bc3-fa5be39a31eb None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:30.984554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5a541-1515-4e97-8bc3-fa5be39a31eb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:30.984678 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5a541-1515-4e97-8bc3-fa5be39a31eb None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:30.986470 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 696/1392] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:30.991831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111a051-692b-4cce-9739-f7c0ae449977 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:30.992339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111a051-692b-4cce-9739-f7c0ae449977 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:30.992453 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111a051-692b-4cce-9739-f7c0ae449977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:31.011655 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d111a051-692b-4cce-9739-f7c0ae449977 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:31.012851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d111a051-692b-4cce-9739-f7c0ae449977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:31.022172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 697/1393] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:31.067613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5a541-1515-4e97-8bc3-fa5be39a31eb None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:31.083851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5a541-1515-4e97-8bc3-fa5be39a31eb None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:31.089266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04059c12-1c2d-4689-b15d-92b016618c30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:31.089907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04059c12-1c2d-4689-b15d-92b016618c30 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:31.090185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04059c12-1c2d-4689-b15d-92b016618c30 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:31.090454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04059c12-1c2d-4689-b15d-92b016618c30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:31.100004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04059c12-1c2d-4689-b15d-92b016618c30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:31.101653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04059c12-1c2d-4689-b15d-92b016618c30 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:31.112284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 697/1394] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:28:32.922186 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc205a-4ef4-4db7-ac16-76eb3a2ff26b None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:32.922758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc205a-4ef4-4db7-ac16-76eb3a2ff26b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:32.922758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc205a-4ef4-4db7-ac16-76eb3a2ff26b None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:32.924136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 698/1395] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:32.929640 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3bbba-0b27-486b-ac56-f88932cb40c9 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:32.930109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3bbba-0b27-486b-ac56-f88932cb40c9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:32.930293 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3bbba-0b27-486b-ac56-f88932cb40c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:32.936250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=52 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:28:32.949355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-43c3bbba-0b27-486b-ac56-f88932cb40c9 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:32.950561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-43c3bbba-0b27-486b-ac56-f88932cb40c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:32.959305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 698/1396] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:33.018202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fb7a8-d468-4bbb-bbc5-4d9ea520b308 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:33.018807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fb7a8-d468-4bbb-bbc5-4d9ea520b308 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:33.019630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fb7a8-d468-4bbb-bbc5-4d9ea520b308 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:33.022016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 699/1397] 10.222.0.39 () {54 vars in 962 bytes} [Tue Jun 7 08:28:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:33.027473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf6928-d934-427d-b1a7-5292fcb0b5ed None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:33.027762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf6928-d934-427d-b1a7-5292fcb0b5ed None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:33.028038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf6928-d934-427d-b1a7-5292fcb0b5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:33.078627 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-01cf6928-d934-427d-b1a7-5292fcb0b5ed None None] MFA Rules not processed for user `d1e4531ac696445ba5bfa9870984befd`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:33.079869 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-01cf6928-d934-427d-b1a7-5292fcb0b5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:33.147503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 699/1398] 10.222.0.39 () {58 vars in 1059 bytes} [Tue Jun 7 08:28:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:33.155418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fb7a8-d468-4bbb-bbc5-4d9ea520b308 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:33.167345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fb7a8-d468-4bbb-bbc5-4d9ea520b308 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:33.171154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00849323-6956-4b1a-adc7-92037240c96d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e4531ac696445ba5bfa9870984befd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:33.172006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00849323-6956-4b1a-adc7-92037240c96d service glance] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:33.172448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00849323-6956-4b1a-adc7-92037240c96d service glance] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:33.172794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00849323-6956-4b1a-adc7-92037240c96d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:33.182567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00849323-6956-4b1a-adc7-92037240c96d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:33.184096 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00849323-6956-4b1a-adc7-92037240c96d service glance] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:33.195638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 700/1399] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:28:33.536756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a49981-24d3-4cca-be21-6692c84e5d73 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:33.537071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a49981-24d3-4cca-be21-6692c84e5d73 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:33.537442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a49981-24d3-4cca-be21-6692c84e5d73 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:33.538807 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 700/1400] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:33.544125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338c7d3-561e-41e8-bc55-c4e65b2d05e8 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:33.544417 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338c7d3-561e-41e8-bc55-c4e65b2d05e8 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:33.544701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338c7d3-561e-41e8-bc55-c4e65b2d05e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:33.563950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-9338c7d3-561e-41e8-bc55-c4e65b2d05e8 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:33.565176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9338c7d3-561e-41e8-bc55-c4e65b2d05e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:33.573006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 701/1401] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:28:33.619398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a49981-24d3-4cca-be21-6692c84e5d73 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:33.634597 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a49981-24d3-4cca-be21-6692c84e5d73 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:33.640248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c70491-8b2d-426a-9262-e4ca5e6b7ffd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:33.641290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c70491-8b2d-426a-9262-e4ca5e6b7ffd service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:33.641793 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c70491-8b2d-426a-9262-e4ca5e6b7ffd service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:33.642172 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c70491-8b2d-426a-9262-e4ca5e6b7ffd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:33.651088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c70491-8b2d-426a-9262-e4ca5e6b7ffd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:33.652678 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c70491-8b2d-426a-9262-e4ca5e6b7ffd service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:33.664051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 701/1402] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:28:33.744929 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceae958-9171-4db9-81e8-cf3b4634db40 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:33.745482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceae958-9171-4db9-81e8-cf3b4634db40 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:33.745842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceae958-9171-4db9-81e8-cf3b4634db40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:33.804601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-dceae958-9171-4db9-81e8-cf3b4634db40 None None] MFA Rules not processed for user `26d6eb06b23b4db88261ae58d650736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:33.806022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-dceae958-9171-4db9-81e8-cf3b4634db40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:33.857586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:28:33.881567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 702/1403] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jun 7 08:28:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:33.926874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df95393-8e05-424e-af3b-b59679d07efd None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:33.927258 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df95393-8e05-424e-af3b-b59679d07efd None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:33.927781 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df95393-8e05-424e-af3b-b59679d07efd None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:33.929407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 702/1404] 10.222.0.39 () {54 vars in 942 bytes} [Tue Jun 7 08:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:33.936182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a2c89-65ed-4c0d-b71c-d24830e6600e None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:33.936686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a2c89-65ed-4c0d-b71c-d24830e6600e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:33.936786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a2c89-65ed-4c0d-b71c-d24830e6600e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:33.996349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-195a2c89-65ed-4c0d-b71c-d24830e6600e None None] MFA Rules not processed for user `586ddffa77a54521b42b2aa50f16cb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:33.997525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-195a2c89-65ed-4c0d-b71c-d24830e6600e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:34.073839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 703/1405] 10.222.0.39 () {58 vars in 1039 bytes} [Tue Jun 7 08:28:33 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:34.083487 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df95393-8e05-424e-af3b-b59679d07efd None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:34.097407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df95393-8e05-424e-af3b-b59679d07efd None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:34.102287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8211aa8-df97-4867-8acf-5bc822c94d34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586ddffa77a54521b42b2aa50f16cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:34.102926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211aa8-df97-4867-8acf-5bc822c94d34 service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:34.103234 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211aa8-df97-4867-8acf-5bc822c94d34 service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:34.103506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211aa8-df97-4867-8acf-5bc822c94d34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:34.112701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8211aa8-df97-4867-8acf-5bc822c94d34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:34.114317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8211aa8-df97-4867-8acf-5bc822c94d34 service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:34.122621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 703/1406] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jun 7 08:28:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:28:36.130087 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bda94-8da9-43b9-935b-0a818842bfc3 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:36.130633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bda94-8da9-43b9-935b-0a818842bfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:36.130633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bda94-8da9-43b9-935b-0a818842bfc3 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:36.132032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 704/1407] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:36.138136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a888eb-7c2e-4300-a283-0c09aaa4e791 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:36.138702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a888eb-7c2e-4300-a283-0c09aaa4e791 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:36.138702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a888eb-7c2e-4300-a283-0c09aaa4e791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:36.158140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-26a888eb-7c2e-4300-a283-0c09aaa4e791 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:36.159372 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-26a888eb-7c2e-4300-a283-0c09aaa4e791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:36.167810 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 704/1408] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:36.213800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bda94-8da9-43b9-935b-0a818842bfc3 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:28:36.230241 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bda94-8da9-43b9-935b-0a818842bfc3 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:28:36.236748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f987959-1f49-4329-99af-7073c3e09d00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:28:36.237460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f987959-1f49-4329-99af-7073c3e09d00 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:36.237751 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f987959-1f49-4329-99af-7073c3e09d00 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:36.238032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f987959-1f49-4329-99af-7073c3e09d00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:36.248966 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f987959-1f49-4329-99af-7073c3e09d00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:28:36.250821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f987959-1f49-4329-99af-7073c3e09d00 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:28:36.261322 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 705/1409] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:28:46.154168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=55 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:28:46.256533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=57 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:28:58.013596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2171c7d-4abc-4190-b889-a3174f1c085f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:58.014196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2171c7d-4abc-4190-b889-a3174f1c085f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:58.014196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2171c7d-4abc-4190-b889-a3174f1c085f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:58.015985 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 705/1410] 10.222.0.39 () {54 vars in 918 bytes} [Tue Jun 7 08:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:58.021293 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e519eb-dced-4181-82dc-0f15b70b0087 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:58.021796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e519eb-dced-4181-82dc-0f15b70b0087 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:58.021927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e519eb-dced-4181-82dc-0f15b70b0087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:58.040091 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-a1e519eb-dced-4181-82dc-0f15b70b0087 None None] MFA Rules not processed for user `7a7d1e8a0f4b4511a7dc8b27651744c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:58.041389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a1e519eb-dced-4181-82dc-0f15b70b0087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:58.050260 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 706/1411] 10.222.0.39 () {58 vars in 1015 bytes} [Tue Jun 7 08:28:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:28:58.064953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cf12c-6bed-42b1-8d45-a64fc5295313 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:58.065272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cf12c-6bed-42b1-8d45-a64fc5295313 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:58.065554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cf12c-6bed-42b1-8d45-a64fc5295313 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:58.066904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 706/1412] 10.222.0.39 () {54 vars in 918 bytes} [Tue Jun 7 08:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:28:58.071724 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb35f-c147-4ff5-b50b-2cb67141b22b None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:28:58.072020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb35f-c147-4ff5-b50b-2cb67141b22b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:28:58.072303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb35f-c147-4ff5-b50b-2cb67141b22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:28:58.090727 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e30cb35f-c147-4ff5-b50b-2cb67141b22b None None] MFA Rules not processed for user `3f2a2bf913d1412f996acf4401ba7673`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:28:58.091918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e30cb35f-c147-4ff5-b50b-2cb67141b22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:28:58.101210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 707/1413] 10.222.0.39 () {58 vars in 1015 bytes} [Tue Jun 7 08:28:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:29:00.890933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2e0cf-8833-4c6d-8f78-653b08ab5d20 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:00.891509 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2e0cf-8833-4c6d-8f78-653b08ab5d20 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:00.891509 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2e0cf-8833-4c6d-8f78-653b08ab5d20 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:00.892877 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:00.898643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf5a7c-f62b-46f5-ac2e-14913cc7f056 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:00.899029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf5a7c-f62b-46f5-ac2e-14913cc7f056 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:00.899275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf5a7c-f62b-46f5-ac2e-14913cc7f056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:00.917596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-adcf5a7c-f62b-46f5-ac2e-14913cc7f056 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:00.918779 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-adcf5a7c-f62b-46f5-ac2e-14913cc7f056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:00.927794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:00.971477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2e0cf-8833-4c6d-8f78-653b08ab5d20 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:00.987477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2e0cf-8833-4c6d-8f78-653b08ab5d20 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:00.992949 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800aaa5d-1ef3-4dc9-af8f-bb0a87a4edbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:00.993619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800aaa5d-1ef3-4dc9-af8f-bb0a87a4edbb service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:00.993919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800aaa5d-1ef3-4dc9-af8f-bb0a87a4edbb service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:00.994184 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800aaa5d-1ef3-4dc9-af8f-bb0a87a4edbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:01.002420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800aaa5d-1ef3-4dc9-af8f-bb0a87a4edbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:01.003988 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800aaa5d-1ef3-4dc9-af8f-bb0a87a4edbb service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:01.013546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 708/1416] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:02.020882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adbafc-68c1-4cc8-b53e-c9ccfe046f1c None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:02.021455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adbafc-68c1-4cc8-b53e-c9ccfe046f1c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:02.021455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adbafc-68c1-4cc8-b53e-c9ccfe046f1c None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:02.022829 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:02.027856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059ea98-e989-45b0-bebe-ba693bf62be2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:02.028246 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059ea98-e989-45b0-bebe-ba693bf62be2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:02.028414 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059ea98-e989-45b0-bebe-ba693bf62be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:02.046870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3059ea98-e989-45b0-bebe-ba693bf62be2 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:02.048079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3059ea98-e989-45b0-bebe-ba693bf62be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:02.055917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:29:02.231984 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adbafc-68c1-4cc8-b53e-c9ccfe046f1c None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:02.247526 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adbafc-68c1-4cc8-b53e-c9ccfe046f1c None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:02.253689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37662f7a-ec85-49ba-a36b-0028336ecc5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:02.254334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37662f7a-ec85-49ba-a36b-0028336ecc5f service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:02.254623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37662f7a-ec85-49ba-a36b-0028336ecc5f service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:02.254907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37662f7a-ec85-49ba-a36b-0028336ecc5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:02.263195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37662f7a-ec85-49ba-a36b-0028336ecc5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:02.264929 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37662f7a-ec85-49ba-a36b-0028336ecc5f service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:02.274704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 710/1419] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:03.214001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb38a12-2f78-4e2c-9561-00df18583ebe None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:03.214560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb38a12-2f78-4e2c-9561-00df18583ebe None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:03.214560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb38a12-2f78-4e2c-9561-00df18583ebe None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:03.215829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:03.221513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baa208-415d-457b-a43b-79c4bae080d7 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:03.221958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baa208-415d-457b-a43b-79c4bae080d7 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:03.222072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baa208-415d-457b-a43b-79c4bae080d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:03.240585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-88baa208-415d-457b-a43b-79c4bae080d7 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:03.241821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-88baa208-415d-457b-a43b-79c4bae080d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:03.249915 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:03.401690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb38a12-2f78-4e2c-9561-00df18583ebe None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:03.416855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb38a12-2f78-4e2c-9561-00df18583ebe None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:03.422898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2b253-668b-4115-a01b-913baac2e06c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:03.423572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2b253-668b-4115-a01b-913baac2e06c service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:03.423882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2b253-668b-4115-a01b-913baac2e06c service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:03.424182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2b253-668b-4115-a01b-913baac2e06c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:03.432646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2b253-668b-4115-a01b-913baac2e06c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:03.434246 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2b253-668b-4115-a01b-913baac2e06c service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:03.444142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 711/1422] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:04.362704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ae631-d570-49f7-b12d-940851b97427 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:04.363239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ae631-d570-49f7-b12d-940851b97427 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:04.363239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ae631-d570-49f7-b12d-940851b97427 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:04.364567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:04.369446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfcd5f-0f9b-4e3d-8fad-b327cc30262d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:04.369852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfcd5f-0f9b-4e3d-8fad-b327cc30262d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:04.370033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfcd5f-0f9b-4e3d-8fad-b327cc30262d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:04.388458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-98bfcd5f-0f9b-4e3d-8fad-b327cc30262d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:04.389728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-98bfcd5f-0f9b-4e3d-8fad-b327cc30262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:04.397976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:04.547991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ae631-d570-49f7-b12d-940851b97427 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:04.562833 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ae631-d570-49f7-b12d-940851b97427 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:04.568177 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f98381-1c2c-413a-8834-5de2684289a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:04.568812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f98381-1c2c-413a-8834-5de2684289a7 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:04.569111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f98381-1c2c-413a-8834-5de2684289a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:04.569408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f98381-1c2c-413a-8834-5de2684289a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:04.577526 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f98381-1c2c-413a-8834-5de2684289a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:04.579105 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f98381-1c2c-413a-8834-5de2684289a7 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:04.588694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 713/1425] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:05.517287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c58d3d-1d14-418f-af85-dce6d4efcc14 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:05.517873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c58d3d-1d14-418f-af85-dce6d4efcc14 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:05.517975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c58d3d-1d14-418f-af85-dce6d4efcc14 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:05.519367 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:05.524640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bf736-589e-4f2b-9058-0894394f1697 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:05.525063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bf736-589e-4f2b-9058-0894394f1697 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:05.525227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bf736-589e-4f2b-9058-0894394f1697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:05.543792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-ef9bf736-589e-4f2b-9058-0894394f1697 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:05.545015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ef9bf736-589e-4f2b-9058-0894394f1697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:05.553849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:05.709447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c58d3d-1d14-418f-af85-dce6d4efcc14 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:05.724121 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c58d3d-1d14-418f-af85-dce6d4efcc14 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:05.729527 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204fdd7-16ec-4016-b212-853b6ce99734 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:05.730167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204fdd7-16ec-4016-b212-853b6ce99734 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:05.730449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204fdd7-16ec-4016-b212-853b6ce99734 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:05.730732 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204fdd7-16ec-4016-b212-853b6ce99734 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:05.739533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204fdd7-16ec-4016-b212-853b6ce99734 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:05.741123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204fdd7-16ec-4016-b212-853b6ce99734 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:05.751347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 714/1428] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:06.623693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe46402-3b22-42a8-88bd-88a29da5d7ff None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:06.624278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe46402-3b22-42a8-88bd-88a29da5d7ff None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:06.624759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe46402-3b22-42a8-88bd-88a29da5d7ff None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:06.626078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:29:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:06.631090 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55051e-d280-4710-8738-93f0754a3644 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:06.631531 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55051e-d280-4710-8738-93f0754a3644 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:06.631658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55051e-d280-4710-8738-93f0754a3644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:06.650078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0d55051e-d280-4710-8738-93f0754a3644 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:06.651280 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0d55051e-d280-4710-8738-93f0754a3644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:06.660008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:06.817768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe46402-3b22-42a8-88bd-88a29da5d7ff None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:06.834792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe46402-3b22-42a8-88bd-88a29da5d7ff None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:06.840791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2bd9d-6aa6-478e-abf2-2cac1e0beee7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:06.841473 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2bd9d-6aa6-478e-abf2-2cac1e0beee7 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:06.841764 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2bd9d-6aa6-478e-abf2-2cac1e0beee7 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:06.842053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2bd9d-6aa6-478e-abf2-2cac1e0beee7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:06.851473 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2bd9d-6aa6-478e-abf2-2cac1e0beee7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:06.853178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2bd9d-6aa6-478e-abf2-2cac1e0beee7 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:06.862684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 716/1431] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:07.724221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1d308-0935-47db-9e31-6ba7da89dfda None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:07.724735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1d308-0935-47db-9e31-6ba7da89dfda None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:07.724858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1d308-0935-47db-9e31-6ba7da89dfda None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:07.726200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:07.731043 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a51646-a8cb-4183-b380-c29750849746 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:07.731475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a51646-a8cb-4183-b380-c29750849746 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:07.731610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a51646-a8cb-4183-b380-c29750849746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:07.750102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e8a51646-a8cb-4183-b380-c29750849746 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:07.751314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e8a51646-a8cb-4183-b380-c29750849746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:07.759224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:07.913378 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1d308-0935-47db-9e31-6ba7da89dfda None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:07.928231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1d308-0935-47db-9e31-6ba7da89dfda None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:07.933713 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1dc801-b949-46e2-b49b-9e76c17cde06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:07.934395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dc801-b949-46e2-b49b-9e76c17cde06 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:07.934623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dc801-b949-46e2-b49b-9e76c17cde06 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:07.934897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dc801-b949-46e2-b49b-9e76c17cde06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:07.943541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1dc801-b949-46e2-b49b-9e76c17cde06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:07.945109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1dc801-b949-46e2-b49b-9e76c17cde06 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:07.955142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 717/1434] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:08.829405 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0012ee-ff59-4a0e-9e24-24b5221e0ef1 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:08.830022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0012ee-ff59-4a0e-9e24-24b5221e0ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:08.830022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0012ee-ff59-4a0e-9e24-24b5221e0ef1 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:08.831615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:08.836654 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a1228-58bb-4667-ae26-af3f758025f7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:08.837122 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a1228-58bb-4667-ae26-af3f758025f7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:08.837225 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a1228-58bb-4667-ae26-af3f758025f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:08.855410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0f8a1228-58bb-4667-ae26-af3f758025f7 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:08.856600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0f8a1228-58bb-4667-ae26-af3f758025f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:08.864959 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:09.020326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0012ee-ff59-4a0e-9e24-24b5221e0ef1 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:09.035449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0012ee-ff59-4a0e-9e24-24b5221e0ef1 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:09.040904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e6268-be90-419c-8523-a13ac3252ab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:09.041569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e6268-be90-419c-8523-a13ac3252ab5 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:09.041855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e6268-be90-419c-8523-a13ac3252ab5 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:09.042132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e6268-be90-419c-8523-a13ac3252ab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:09.051111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e6268-be90-419c-8523-a13ac3252ab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:09.052776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e6268-be90-419c-8523-a13ac3252ab5 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:09.063231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 719/1437] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:09.974917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2357da-5614-43f6-93bb-9d872471774b None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:09.975424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2357da-5614-43f6-93bb-9d872471774b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:09.975424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2357da-5614-43f6-93bb-9d872471774b None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:09.976795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:09.981957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08f4d5-8921-4c99-b25b-c0097e942631 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:09.982503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08f4d5-8921-4c99-b25b-c0097e942631 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:09.982503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08f4d5-8921-4c99-b25b-c0097e942631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:10.001820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-5e08f4d5-8921-4c99-b25b-c0097e942631 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:10.003030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5e08f4d5-8921-4c99-b25b-c0097e942631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:10.010866 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:10.162178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2357da-5614-43f6-93bb-9d872471774b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:10.176575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2357da-5614-43f6-93bb-9d872471774b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:10.182308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89021ddb-3b89-4430-ae47-87e8f9e69afd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:10.182981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89021ddb-3b89-4430-ae47-87e8f9e69afd service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:10.183300 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89021ddb-3b89-4430-ae47-87e8f9e69afd service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:10.183579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89021ddb-3b89-4430-ae47-87e8f9e69afd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:10.192809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89021ddb-3b89-4430-ae47-87e8f9e69afd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:10.194420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89021ddb-3b89-4430-ae47-87e8f9e69afd service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:10.204996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 720/1440] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:12.342529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d052e-76ed-49b5-a0ff-1516a9d73f1b None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:12.343116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d052e-76ed-49b5-a0ff-1516a9d73f1b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:12.343239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d052e-76ed-49b5-a0ff-1516a9d73f1b None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:12.344554 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:12.349981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda445ca-7e12-4a3c-bac0-01aa9f0cd86f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:12.350416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda445ca-7e12-4a3c-bac0-01aa9f0cd86f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:12.350573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda445ca-7e12-4a3c-bac0-01aa9f0cd86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:12.368530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-fda445ca-7e12-4a3c-bac0-01aa9f0cd86f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:12.369790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fda445ca-7e12-4a3c-bac0-01aa9f0cd86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:12.378344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:12.416918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d052e-76ed-49b5-a0ff-1516a9d73f1b None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:12.427478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d052e-76ed-49b5-a0ff-1516a9d73f1b None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:12.430908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d37117-0e11-454c-a9a7-062f85132f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:12.431533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d37117-0e11-454c-a9a7-062f85132f0f service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:12.431809 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d37117-0e11-454c-a9a7-062f85132f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:12.432083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d37117-0e11-454c-a9a7-062f85132f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:12.441488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d37117-0e11-454c-a9a7-062f85132f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:12.442903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d37117-0e11-454c-a9a7-062f85132f0f service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:12.452668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 722/1443] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:14.957581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9c058-dfea-45bb-b406-5b20e10aec47 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:14.958159 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9c058-dfea-45bb-b406-5b20e10aec47 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:14.958159 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9c058-dfea-45bb-b406-5b20e10aec47 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:14.959601 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:14.964908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749616-0b1e-44e1-97a0-02c25fac80ed None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:14.965352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749616-0b1e-44e1-97a0-02c25fac80ed None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:14.965512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749616-0b1e-44e1-97a0-02c25fac80ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:14.984084 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-12749616-0b1e-44e1-97a0-02c25fac80ed None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:14.985321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-12749616-0b1e-44e1-97a0-02c25fac80ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:14.993097 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:15.031267 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9c058-dfea-45bb-b406-5b20e10aec47 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:15.042374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9c058-dfea-45bb-b406-5b20e10aec47 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:15.046529 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98abe97-ba97-4b9a-a926-57b1866af6dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:15.047246 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98abe97-ba97-4b9a-a926-57b1866af6dd service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:15.047575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98abe97-ba97-4b9a-a926-57b1866af6dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:15.047892 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98abe97-ba97-4b9a-a926-57b1866af6dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:15.057063 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98abe97-ba97-4b9a-a926-57b1866af6dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:15.058509 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98abe97-ba97-4b9a-a926-57b1866af6dd service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:15.068674 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 723/1446] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:17.687637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c845c21-f31b-455b-a8c7-cf64ff6368ac None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:17.688217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c845c21-f31b-455b-a8c7-cf64ff6368ac None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:17.688217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c845c21-f31b-455b-a8c7-cf64ff6368ac None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:17.689733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:29:17.695610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8cfad-464a-4bfa-ad17-1625246a142d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:17.696030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8cfad-464a-4bfa-ad17-1625246a142d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:17.696211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8cfad-464a-4bfa-ad17-1625246a142d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:17.715720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f8b8cfad-464a-4bfa-ad17-1625246a142d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:17.717647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f8b8cfad-464a-4bfa-ad17-1625246a142d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:17.726306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:17.765210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c845c21-f31b-455b-a8c7-cf64ff6368ac None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:17.775769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c845c21-f31b-455b-a8c7-cf64ff6368ac None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:17.779948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa415d-a8cf-4d55-a28d-fc0c4df6b4f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:17.780679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa415d-a8cf-4d55-a28d-fc0c4df6b4f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:17.780963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa415d-a8cf-4d55-a28d-fc0c4df6b4f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:17.781276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa415d-a8cf-4d55-a28d-fc0c4df6b4f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:17.790323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa415d-a8cf-4d55-a28d-fc0c4df6b4f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:17.791739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa415d-a8cf-4d55-a28d-fc0c4df6b4f5 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:17.801218 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 725/1449] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:20.227600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7b4b6-1b16-43ee-9127-4e4c74585129 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:20.228121 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7b4b6-1b16-43ee-9127-4e4c74585129 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:20.228121 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7b4b6-1b16-43ee-9127-4e4c74585129 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:20.229478 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:20.235423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7036f0e-2d00-418c-b2cd-4da78ffeca13 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:20.235975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7036f0e-2d00-418c-b2cd-4da78ffeca13 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:20.235975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7036f0e-2d00-418c-b2cd-4da78ffeca13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:20.253491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d7036f0e-2d00-418c-b2cd-4da78ffeca13 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:20.254691 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d7036f0e-2d00-418c-b2cd-4da78ffeca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:20.262393 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:20.302193 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7b4b6-1b16-43ee-9127-4e4c74585129 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:20.313457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7b4b6-1b16-43ee-9127-4e4c74585129 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:20.317001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c048d-4a54-46d7-acb0-a5ed11d78828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:20.317699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c048d-4a54-46d7-acb0-a5ed11d78828 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:20.318014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c048d-4a54-46d7-acb0-a5ed11d78828 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:20.318314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c048d-4a54-46d7-acb0-a5ed11d78828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:20.327895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c048d-4a54-46d7-acb0-a5ed11d78828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:20.329378 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c048d-4a54-46d7-acb0-a5ed11d78828 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:20.339128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 726/1452] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:22.697218 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b9092-cac5-41c2-bec3-877de33fc3fa None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:22.697817 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b9092-cac5-41c2-bec3-877de33fc3fa None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:22.697817 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b9092-cac5-41c2-bec3-877de33fc3fa None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:22.699198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:22.705524 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0c57a-f169-46b1-ba1f-738c8f123c70 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:22.705996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0c57a-f169-46b1-ba1f-738c8f123c70 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:22.706104 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0c57a-f169-46b1-ba1f-738c8f123c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:22.724287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-32d0c57a-f169-46b1-ba1f-738c8f123c70 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:22.725537 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-32d0c57a-f169-46b1-ba1f-738c8f123c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:22.733479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:22.772284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b9092-cac5-41c2-bec3-877de33fc3fa None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:22.783468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b9092-cac5-41c2-bec3-877de33fc3fa None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:22.787049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae28d3-a90b-4dfc-933f-1cae5ebd7178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:22.787677 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae28d3-a90b-4dfc-933f-1cae5ebd7178 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:22.787958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae28d3-a90b-4dfc-933f-1cae5ebd7178 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:22.788231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae28d3-a90b-4dfc-933f-1cae5ebd7178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:22.797147 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae28d3-a90b-4dfc-933f-1cae5ebd7178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:22.798595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae28d3-a90b-4dfc-933f-1cae5ebd7178 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:22.807840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 728/1455] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:25.162449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec12c-7156-439b-ac89-56000791b28d None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:25.163028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec12c-7156-439b-ac89-56000791b28d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:25.163028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec12c-7156-439b-ac89-56000791b28d None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:25.164290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:25.170416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e88e0-0f0e-4ee5-abb0-ae3bc0b9ad89 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:25.171197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e88e0-0f0e-4ee5-abb0-ae3bc0b9ad89 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:25.171197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e88e0-0f0e-4ee5-abb0-ae3bc0b9ad89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:25.189292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-814e88e0-0f0e-4ee5-abb0-ae3bc0b9ad89 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:25.190506 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-814e88e0-0f0e-4ee5-abb0-ae3bc0b9ad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:25.198513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:25.238667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cec12c-7156-439b-ac89-56000791b28d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:25.250347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cec12c-7156-439b-ac89-56000791b28d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:25.254405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc62b0-f90e-4f59-98b3-d64361f88f9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:25.255078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc62b0-f90e-4f59-98b3-d64361f88f9a service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:25.255360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc62b0-f90e-4f59-98b3-d64361f88f9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:25.255634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc62b0-f90e-4f59-98b3-d64361f88f9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:25.265444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc62b0-f90e-4f59-98b3-d64361f88f9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:25.266856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc62b0-f90e-4f59-98b3-d64361f88f9a service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:25.276421 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 729/1458] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:27.675428 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161231c-a367-4c06-8096-1e7076749a48 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:27.676039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161231c-a367-4c06-8096-1e7076749a48 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:27.676039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161231c-a367-4c06-8096-1e7076749a48 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:27.677484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:27.683702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce3bf0-b8b3-47ce-81e9-d4a80170516e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:27.684302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce3bf0-b8b3-47ce-81e9-d4a80170516e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:27.684302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce3bf0-b8b3-47ce-81e9-d4a80170516e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:27.690603 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=70 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:27.706182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-adce3bf0-b8b3-47ce-81e9-d4a80170516e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:27.707475 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-adce3bf0-b8b3-47ce-81e9-d4a80170516e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:27.717676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:27.758946 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161231c-a367-4c06-8096-1e7076749a48 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:27.760798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d161231c-a367-4c06-8096-1e7076749a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:27.769312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=73 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:27.776127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161231c-a367-4c06-8096-1e7076749a48 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:27.781281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e5675-b15a-437f-96b3-533990777673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:27.781961 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e5675-b15a-437f-96b3-533990777673 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:27.782240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e5675-b15a-437f-96b3-533990777673 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:27.782513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e5675-b15a-437f-96b3-533990777673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:27.792810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e5675-b15a-437f-96b3-533990777673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:27.794327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e5675-b15a-437f-96b3-533990777673 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:27.805644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 731/1461] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:30.244470 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64355db3-9005-423e-b6f7-b93612aae450 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:30.245049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64355db3-9005-423e-b6f7-b93612aae450 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:30.245049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64355db3-9005-423e-b6f7-b93612aae450 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:30.246448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:30.252815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047617c-a06a-4e23-9b8e-ff43bab744f6 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:30.253285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047617c-a06a-4e23-9b8e-ff43bab744f6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:30.253427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047617c-a06a-4e23-9b8e-ff43bab744f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:30.271567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-b047617c-a06a-4e23-9b8e-ff43bab744f6 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:30.272775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b047617c-a06a-4e23-9b8e-ff43bab744f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:30.280794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:30.324472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64355db3-9005-423e-b6f7-b93612aae450 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:30.336285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64355db3-9005-423e-b6f7-b93612aae450 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:30.339814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929ff78-2955-4420-82fd-b6eb80ec271b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:30.340453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929ff78-2955-4420-82fd-b6eb80ec271b service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:30.340732 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929ff78-2955-4420-82fd-b6eb80ec271b service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:30.341005 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929ff78-2955-4420-82fd-b6eb80ec271b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:30.350412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929ff78-2955-4420-82fd-b6eb80ec271b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:30.351881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929ff78-2955-4420-82fd-b6eb80ec271b service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:30.361945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 732/1464] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:32.713714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065de15-44d7-4ec7-85a9-015158687296 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:32.714345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065de15-44d7-4ec7-85a9-015158687296 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:32.714345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065de15-44d7-4ec7-85a9-015158687296 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:32.715667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:29:32.721934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326cad9-b5cd-4218-b7a0-3f77a5ca2000 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:32.722560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326cad9-b5cd-4218-b7a0-3f77a5ca2000 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:32.722560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326cad9-b5cd-4218-b7a0-3f77a5ca2000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:32.743714 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c326cad9-b5cd-4218-b7a0-3f77a5ca2000 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:32.744937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c326cad9-b5cd-4218-b7a0-3f77a5ca2000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:32.753983 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:32.791153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065de15-44d7-4ec7-85a9-015158687296 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:32.803026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065de15-44d7-4ec7-85a9-015158687296 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:32.806562 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6378be1-f900-425e-bb50-9bdfa5f98b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:32.807201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6378be1-f900-425e-bb50-9bdfa5f98b1d service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:32.807486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6378be1-f900-425e-bb50-9bdfa5f98b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:32.807765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6378be1-f900-425e-bb50-9bdfa5f98b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:32.818013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6378be1-f900-425e-bb50-9bdfa5f98b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:32.819472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6378be1-f900-425e-bb50-9bdfa5f98b1d service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:32.829660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 734/1467] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:29:35.338292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015cb23-e569-4c79-9d4f-622ec4f2dcd5 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:35.338839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015cb23-e569-4c79-9d4f-622ec4f2dcd5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:35.338839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015cb23-e569-4c79-9d4f-622ec4f2dcd5 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:35.340164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:29:35.346404 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6c8f3-30ed-49bb-ae89-b38ee7ba24d2 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:35.346876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6c8f3-30ed-49bb-ae89-b38ee7ba24d2 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:35.346972 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6c8f3-30ed-49bb-ae89-b38ee7ba24d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:35.365578 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-41c6c8f3-30ed-49bb-ae89-b38ee7ba24d2 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:35.366798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-41c6c8f3-30ed-49bb-ae89-b38ee7ba24d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:35.375530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:35.413771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015cb23-e569-4c79-9d4f-622ec4f2dcd5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:35.424742 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015cb23-e569-4c79-9d4f-622ec4f2dcd5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:35.428381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e3536-0d21-4f9e-b737-935cdff8a473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:35.429021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e3536-0d21-4f9e-b737-935cdff8a473 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:35.429351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e3536-0d21-4f9e-b737-935cdff8a473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:35.429630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e3536-0d21-4f9e-b737-935cdff8a473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:35.439852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e3536-0d21-4f9e-b737-935cdff8a473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:35.441327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e3536-0d21-4f9e-b737-935cdff8a473 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:35.451704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 735/1470] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:45.360248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=17 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:45.446624 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=17 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:30:07.998881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b450a7c-8083-460f-b06f-23812fdd59d6 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:07.999481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b450a7c-8083-460f-b06f-23812fdd59d6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:07.999481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b450a7c-8083-460f-b06f-23812fdd59d6 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:08.000741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 736/1471] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:08.006208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cac96-b589-45d5-b16a-8eb48e426ee7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:08.006718 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cac96-b589-45d5-b16a-8eb48e426ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:08.006821 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cac96-b589-45d5-b16a-8eb48e426ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:08.024612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-483cac96-b589-45d5-b16a-8eb48e426ee7 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:08.025847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-483cac96-b589-45d5-b16a-8eb48e426ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:08.034022 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 736/1472] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:30:08.083323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8acb8-5aee-4fde-a91d-4f7319df2537 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:08.083686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8acb8-5aee-4fde-a91d-4f7319df2537 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:08.083905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8acb8-5aee-4fde-a91d-4f7319df2537 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:08.085300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 737/1473] 10.222.0.39 () {54 vars in 962 bytes} [Tue Jun 7 08:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:08.090100 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d5f4a-c155-44f0-9e33-6d7b0c84df9c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:08.090398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d5f4a-c155-44f0-9e33-6d7b0c84df9c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:08.090685 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d5f4a-c155-44f0-9e33-6d7b0c84df9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:08.109349 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c39d5f4a-c155-44f0-9e33-6d7b0c84df9c None None] MFA Rules not processed for user `d1e4531ac696445ba5bfa9870984befd`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:08.110574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c39d5f4a-c155-44f0-9e33-6d7b0c84df9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:08.117068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 737/1474] 10.222.0.39 () {58 vars in 1059 bytes} [Tue Jun 7 08:30:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:08.125340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8acb8-5aee-4fde-a91d-4f7319df2537 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:08.136265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8acb8-5aee-4fde-a91d-4f7319df2537 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:08.140259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb7ace-6858-4bc0-aa71-7b2326ae9c10 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e4531ac696445ba5bfa9870984befd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:08.140904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb7ace-6858-4bc0-aa71-7b2326ae9c10 service glance] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:08.141226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb7ace-6858-4bc0-aa71-7b2326ae9c10 service glance] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:08.141510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb7ace-6858-4bc0-aa71-7b2326ae9c10 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:08.151003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb7ace-6858-4bc0-aa71-7b2326ae9c10 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:08.152477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb7ace-6858-4bc0-aa71-7b2326ae9c10 service glance] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:08.162277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 738/1475] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:10.608955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2353df-3627-41fe-b127-dd7a75e9013c None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:10.609586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2353df-3627-41fe-b127-dd7a75e9013c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:10.609586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2353df-3627-41fe-b127-dd7a75e9013c None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:10.610932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 738/1476] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:10.616511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f95a0-04c4-48ee-9792-957f7708bf2e None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:10.616962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f95a0-04c4-48ee-9792-957f7708bf2e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:10.617074 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f95a0-04c4-48ee-9792-957f7708bf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:10.635655 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d90f95a0-04c4-48ee-9792-957f7708bf2e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:10.636873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d90f95a0-04c4-48ee-9792-957f7708bf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:10.645875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 739/1477] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:10.669721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2353df-3627-41fe-b127-dd7a75e9013c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:10.682268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2353df-3627-41fe-b127-dd7a75e9013c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:10.687466 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89d9cc-d26d-44a2-8c4c-4fd7155864c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:10.688119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89d9cc-d26d-44a2-8c4c-4fd7155864c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:10.688408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89d9cc-d26d-44a2-8c4c-4fd7155864c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:10.688681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89d9cc-d26d-44a2-8c4c-4fd7155864c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:10.689466 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89d9cc-d26d-44a2-8c4c-4fd7155864c7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:10.691265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-ed89d9cc-d26d-44a2-8c4c-4fd7155864c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:30:10.691265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Jun 07 08:30:10.691265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: Jun 07 08:30:10.692412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89d9cc-d26d-44a2-8c4c-4fd7155864c7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:10.713371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 739/1478] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jun 7 08:30:10 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:30:13.056689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63940585-00e6-43a0-a7a8-de136eacc722 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:13.057281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63940585-00e6-43a0-a7a8-de136eacc722 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:13.057407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63940585-00e6-43a0-a7a8-de136eacc722 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:13.059825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 740/1479] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:30:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:13.066015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a28c34-7918-4689-a996-90fe1ffc966a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:13.066663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a28c34-7918-4689-a996-90fe1ffc966a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:13.066663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a28c34-7918-4689-a996-90fe1ffc966a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:13.087736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-85a28c34-7918-4689-a996-90fe1ffc966a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:13.088979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-85a28c34-7918-4689-a996-90fe1ffc966a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:13.097484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 740/1480] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:13.144066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63940585-00e6-43a0-a7a8-de136eacc722 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:13.160612 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63940585-00e6-43a0-a7a8-de136eacc722 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:13.167111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4fb04-9978-4a18-ae07-ce653183be24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:13.167783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4fb04-9978-4a18-ae07-ce653183be24 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:13.168067 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4fb04-9978-4a18-ae07-ce653183be24 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:13.168386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4fb04-9978-4a18-ae07-ce653183be24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:13.185225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4fb04-9978-4a18-ae07-ce653183be24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:13.188265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4fb04-9978-4a18-ae07-ce653183be24 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:13.204544 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 741/1481] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:15.830506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6c1a6-5f37-4d10-9f1c-bfbc4363bf65 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:15.831435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6c1a6-5f37-4d10-9f1c-bfbc4363bf65 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:15.831435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6c1a6-5f37-4d10-9f1c-bfbc4363bf65 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:15.832699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 741/1482] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:15.838582 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a984c-42d8-4a24-a418-1c2b31ad11a7 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:15.839146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a984c-42d8-4a24-a418-1c2b31ad11a7 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:15.839146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a984c-42d8-4a24-a418-1c2b31ad11a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:15.859950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d45a984c-42d8-4a24-a418-1c2b31ad11a7 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:15.861194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d45a984c-42d8-4a24-a418-1c2b31ad11a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:15.869544 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 742/1483] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:30:15.908769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6c1a6-5f37-4d10-9f1c-bfbc4363bf65 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:15.924023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6c1a6-5f37-4d10-9f1c-bfbc4363bf65 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:15.929968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52d3a9-01f4-4999-9528-ee0ceb1ce3dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:15.931092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52d3a9-01f4-4999-9528-ee0ceb1ce3dc service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:15.931569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52d3a9-01f4-4999-9528-ee0ceb1ce3dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:15.932087 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52d3a9-01f4-4999-9528-ee0ceb1ce3dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:15.950796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52d3a9-01f4-4999-9528-ee0ceb1ce3dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:15.953680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52d3a9-01f4-4999-9528-ee0ceb1ce3dc service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:15.971819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 742/1484] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:30:19.077109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c5fb0-2fb3-43ac-ae2c-de519d81ea0a None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:19.078050 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c5fb0-2fb3-43ac-ae2c-de519d81ea0a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:19.078050 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c5fb0-2fb3-43ac-ae2c-de519d81ea0a None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:19.080248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 743/1485] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:30:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:19.086502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185db9c-a14e-4d64-ab69-04b542ebc886 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:19.087030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185db9c-a14e-4d64-ab69-04b542ebc886 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:19.087030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185db9c-a14e-4d64-ab69-04b542ebc886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:19.108033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f185db9c-a14e-4d64-ab69-04b542ebc886 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:19.109338 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f185db9c-a14e-4d64-ab69-04b542ebc886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:19.118012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 743/1486] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:19.179133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c5fb0-2fb3-43ac-ae2c-de519d81ea0a None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:19.191238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c5fb0-2fb3-43ac-ae2c-de519d81ea0a None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:19.194943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244a4d9-df30-4fd7-9ccd-ad5714ba43e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:19.195599 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244a4d9-df30-4fd7-9ccd-ad5714ba43e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:19.195966 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244a4d9-df30-4fd7-9ccd-ad5714ba43e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:19.196161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244a4d9-df30-4fd7-9ccd-ad5714ba43e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:19.212607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244a4d9-df30-4fd7-9ccd-ad5714ba43e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:19.215186 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244a4d9-df30-4fd7-9ccd-ad5714ba43e3 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:19.234830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 744/1487] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:21.927670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccce584-177c-4855-a208-c6f0bf3f13bf None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:21.928301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccce584-177c-4855-a208-c6f0bf3f13bf None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:21.928301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccce584-177c-4855-a208-c6f0bf3f13bf None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:21.930096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 744/1488] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:30:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:21.937215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4775997-ead1-4cf6-966a-dcb231a43885 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:21.938077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4775997-ead1-4cf6-966a-dcb231a43885 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:21.938077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4775997-ead1-4cf6-966a-dcb231a43885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:21.960322 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d4775997-ead1-4cf6-966a-dcb231a43885 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:21.961652 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d4775997-ead1-4cf6-966a-dcb231a43885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:21.974426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 745/1489] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:22.018545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccce584-177c-4855-a208-c6f0bf3f13bf None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:22.031948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccce584-177c-4855-a208-c6f0bf3f13bf None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:22.035812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867c791-3d17-44fa-908c-01693c10b77c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:22.036476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867c791-3d17-44fa-908c-01693c10b77c service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:22.036762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867c791-3d17-44fa-908c-01693c10b77c service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:22.037038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867c791-3d17-44fa-908c-01693c10b77c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:22.048158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867c791-3d17-44fa-908c-01693c10b77c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:22.049848 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867c791-3d17-44fa-908c-01693c10b77c service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:22.063001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 745/1490] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:25.016997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bde996-b580-44be-aadc-ea2b7a036346 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:25.018039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bde996-b580-44be-aadc-ea2b7a036346 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:25.018039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bde996-b580-44be-aadc-ea2b7a036346 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:25.019719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 746/1491] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jun 7 08:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:25.026473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f01ee-9a9d-46b4-bc97-1f0b63628bc7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:25.027063 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f01ee-9a9d-46b4-bc97-1f0b63628bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:25.028017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f01ee-9a9d-46b4-bc97-1f0b63628bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:25.048480 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-134f01ee-9a9d-46b4-bc97-1f0b63628bc7 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:25.049836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-134f01ee-9a9d-46b4-bc97-1f0b63628bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:25.058171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 746/1492] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:30:25.101303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bde996-b580-44be-aadc-ea2b7a036346 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:25.114397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bde996-b580-44be-aadc-ea2b7a036346 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:25.118186 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edd9fb-610b-49c7-83cc-70d2c98eab39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:25.118853 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edd9fb-610b-49c7-83cc-70d2c98eab39 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:25.119137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edd9fb-610b-49c7-83cc-70d2c98eab39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:25.119412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edd9fb-610b-49c7-83cc-70d2c98eab39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:25.138712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edd9fb-610b-49c7-83cc-70d2c98eab39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:25.140387 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edd9fb-610b-49c7-83cc-70d2c98eab39 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:25.153458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 747/1493] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:35.043537 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=33 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:30:35.147960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=33 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:32:27.479628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf86d1-e54f-40e8-ab0c-86960ae67937 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:27.480334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf86d1-e54f-40e8-ab0c-86960ae67937 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:27.480334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf86d1-e54f-40e8-ab0c-86960ae67937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:27.502068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-99bf86d1-e54f-40e8-ab0c-86960ae67937 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:32:27.503463 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-99bf86d1-e54f-40e8-ab0c-86960ae67937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:27.512086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 747/1494] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:32:27.520669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f698750-1b27-4868-ae1f-3d58e0a0d532 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:27.521374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f698750-1b27-4868-ae1f-3d58e0a0d532 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:27.521374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f698750-1b27-4868-ae1f-3d58e0a0d532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:27.542115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-5f698750-1b27-4868-ae1f-3d58e0a0d532 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:32:27.543826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5f698750-1b27-4868-ae1f-3d58e0a0d532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:27.553307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 748/1495] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:32:27.559202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf86d1-e54f-40e8-ab0c-86960ae67937 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:27.569310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf86d1-e54f-40e8-ab0c-86960ae67937 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:27.574028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd4275-8802-4c85-a774-04f4f9c5ce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:27.574703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd4275-8802-4c85-a774-04f4f9c5ce75 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:27.574982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd4275-8802-4c85-a774-04f4f9c5ce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:27.575258 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd4275-8802-4c85-a774-04f4f9c5ce75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:27.575991 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd4275-8802-4c85-a774-04f4f9c5ce75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:32:27.576297 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd4275-8802-4c85-a774-04f4f9c5ce75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:27.577467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-a2dd4275-8802-4c85-a774-04f4f9c5ce75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:32:27.577467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Jun 07 08:32:27.577467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: Jun 07 08:32:27.578382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd4275-8802-4c85-a774-04f4f9c5ce75 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:27.599245 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 748/1496] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:32:27.609529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f698750-1b27-4868-ae1f-3d58e0a0d532 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:27.621864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f698750-1b27-4868-ae1f-3d58e0a0d532 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:27.629430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4ac73-d746-41a6-a6c5-ad1fbdda751e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:27.630616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4ac73-d746-41a6-a6c5-ad1fbdda751e admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:27.631155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4ac73-d746-41a6-a6c5-ad1fbdda751e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:27.631646 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4ac73-d746-41a6-a6c5-ad1fbdda751e admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:27.632975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4ac73-d746-41a6-a6c5-ad1fbdda751e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:27.636336 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: WARNING py.warnings [None req-fba4ac73-d746-41a6-a6c5-ad1fbdda751e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:32:27.636336 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: warnings.warn(msg) Jun 07 08:32:27.636336 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: Jun 07 08:32:27.638875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4ac73-d746-41a6-a6c5-ad1fbdda751e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:27.688962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 749/1497] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:32:27 2022] POST /identity/v3/projects => generated 379 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:32:27.694121 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd4275-8802-4c85-a774-04f4f9c5ce75 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:27.707132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd4275-8802-4c85-a774-04f4f9c5ce75 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:27.786951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863eddf-de9b-4f36-a2d9-20655c528207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:27.787554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863eddf-de9b-4f36-a2d9-20655c528207 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:27.787837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863eddf-de9b-4f36-a2d9-20655c528207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:27.788110 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863eddf-de9b-4f36-a2d9-20655c528207 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:27.788850 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863eddf-de9b-4f36-a2d9-20655c528207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:27.790515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-e863eddf-de9b-4f36-a2d9-20655c528207 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:32:27.790515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Jun 07 08:32:27.790515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: Jun 07 08:32:27.791768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863eddf-de9b-4f36-a2d9-20655c528207 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:27.819671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 749/1498] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:32:27 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:32:27.824528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4ac73-d746-41a6-a6c5-ad1fbdda751e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:27.836534 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4ac73-d746-41a6-a6c5-ad1fbdda751e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:27.841383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0972b-7a11-407b-915f-5fbf00fe6fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:27.842074 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0972b-7a11-407b-915f-5fbf00fe6fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:27.842360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0972b-7a11-407b-915f-5fbf00fe6fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:27.842639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0972b-7a11-407b-915f-5fbf00fe6fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:27.843383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0972b-7a11-407b-915f-5fbf00fe6fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:27.844422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: WARNING py.warnings [None req-81d0972b-7a11-407b-915f-5fbf00fe6fe1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:32:27.844422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: warnings.warn(msg) Jun 07 08:32:27.844422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: Jun 07 08:32:27.845356 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0972b-7a11-407b-915f-5fbf00fe6fe1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:27.859740 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 750/1499] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:32:27.865727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863eddf-de9b-4f36-a2d9-20655c528207 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:27.876782 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863eddf-de9b-4f36-a2d9-20655c528207 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:27.881651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc153f15-3b73-4ec7-8656-95f657af8ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:27.882335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc153f15-3b73-4ec7-8656-95f657af8ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:27.882623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc153f15-3b73-4ec7-8656-95f657af8ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:27.883525 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc153f15-3b73-4ec7-8656-95f657af8ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:27.884730 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc153f15-3b73-4ec7-8656-95f657af8ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:27.885806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-fc153f15-3b73-4ec7-8656-95f657af8ed5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:32:27.885806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Jun 07 08:32:27.885806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: Jun 07 08:32:27.886727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc153f15-3b73-4ec7-8656-95f657af8ed5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:27.901668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 750/1500] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:32:27.905960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0972b-7a11-407b-915f-5fbf00fe6fe1 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:27.916543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0972b-7a11-407b-915f-5fbf00fe6fe1 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:27.921343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb14b5-9a5b-4b82-a7e6-d3e75b440f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:27.922220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb14b5-9a5b-4b82-a7e6-d3e75b440f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:27.922504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb14b5-9a5b-4b82-a7e6-d3e75b440f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:27.922780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb14b5-9a5b-4b82-a7e6-d3e75b440f66 admin admin] PATH_INFO: `/v3/projects/8fa6531224744ca6a263c86e59c9f953/users/6f806265607548d881b01ab7227e7331/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:27.939864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb14b5-9a5b-4b82-a7e6-d3e75b440f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa6531224744ca6a263c86e59c9f953, user_id=6f806265607548d881b01ab7227e7331, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:27.940850 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: WARNING py.warnings [None req-2aeb14b5-9a5b-4b82-a7e6-d3e75b440f66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:32:27.940850 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: warnings.warn(msg) Jun 07 08:32:27.940850 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: Jun 07 08:32:27.942871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb14b5-9a5b-4b82-a7e6-d3e75b440f66 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:27.957629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 751/1501] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:32:27 2022] PUT /identity/v3/projects/8fa6531224744ca6a263c86e59c9f953/users/6f806265607548d881b01ab7227e7331/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:32:27.971017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bfe6c-3c1a-4b08-a1c9-5c35141dad53 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:27.971308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bfe6c-3c1a-4b08-a1c9-5c35141dad53 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:27.971594 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bfe6c-3c1a-4b08-a1c9-5c35141dad53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:28.000421 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7e3bfe6c-3c1a-4b08-a1c9-5c35141dad53 None None] MFA Rules not processed for user `6f806265607548d881b01ab7227e7331`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:32:28.001681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7e3bfe6c-3c1a-4b08-a1c9-5c35141dad53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:28.089341 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 751/1502] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:32:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:32:28.109508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae135129-5090-44bd-9ac4-21903077de33 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:28.110028 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae135129-5090-44bd-9ac4-21903077de33 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:28.110530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae135129-5090-44bd-9ac4-21903077de33 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:28.113183 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 752/1503] 10.222.0.39 () {54 vars in 963 bytes} [Tue Jun 7 08:32:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:32:28.120809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874bd9d-b676-4e0b-ae1d-df2ad0693173 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:28.121116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874bd9d-b676-4e0b-ae1d-df2ad0693173 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:28.121424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874bd9d-b676-4e0b-ae1d-df2ad0693173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:28.140423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1874bd9d-b676-4e0b-ae1d-df2ad0693173 None None] MFA Rules not processed for user `2b6943045d594d729fc7c4b15cbaa07e`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:32:28.141670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1874bd9d-b676-4e0b-ae1d-df2ad0693173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:28.209824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:32:28.225325 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 752/1504] 10.222.0.39 () {58 vars in 1060 bytes} [Tue Jun 7 08:32:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:32:28.233765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae135129-5090-44bd-9ac4-21903077de33 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:28.245524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae135129-5090-44bd-9ac4-21903077de33 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:28.249498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96b797-87aa-4f08-97b4-94c870ecc419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:28.250069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96b797-87aa-4f08-97b4-94c870ecc419 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:28.250344 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96b797-87aa-4f08-97b4-94c870ecc419 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:28.250618 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96b797-87aa-4f08-97b4-94c870ecc419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:28.259836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96b797-87aa-4f08-97b4-94c870ecc419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:28.261328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96b797-87aa-4f08-97b4-94c870ecc419 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:28.270797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 753/1505] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:32:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:32:28.318736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423e5a2-ad74-4c96-946a-3d89558b7e87 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:28.319031 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423e5a2-ad74-4c96-946a-3d89558b7e87 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:28.319319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423e5a2-ad74-4c96-946a-3d89558b7e87 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:28.320639 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 753/1506] 10.222.0.39 () {52 vars in 829 bytes} [Tue Jun 7 08:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:32:28.325045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96b797-87aa-4f08-97b4-94c870ecc419 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:28.396572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96b797-87aa-4f08-97b4-94c870ecc419 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:28.401690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:28.402581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:28.402867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:28.403145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] PATH_INFO: `/v3/users/6f806265607548d881b01ab7227e7331` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:28.404468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f806265607548d881b01ab7227e7331)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:28.405541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: WARNING py.warnings [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:32:28.405541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: warnings.warn(msg) Jun 07 08:32:28.405541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: Jun 07 08:32:28.406756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:28.484108 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f806265607548d881b01ab7227e7331'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:32:28.485632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f806265607548d881b01ab7227e7331'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:32:28.501845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f806265607548d881b01ab7227e7331'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:32:28.525443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f806265607548d881b01ab7227e7331'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:32:28.526347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f806265607548d881b01ab7227e7331'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:32:28.532263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:32:28.547349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 754/1507] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:32:28 2022] DELETE /identity/v3/users/6f806265607548d881b01ab7227e7331 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:32:28.552037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d760071-2940-4355-9bc0-152568e23a67 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:28.554292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d760071-2940-4355-9bc0-152568e23a67 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:28.554292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d760071-2940-4355-9bc0-152568e23a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:28.572475 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4d760071-2940-4355-9bc0-152568e23a67 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:32:28.573751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4d760071-2940-4355-9bc0-152568e23a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:28.644710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 754/1508] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:32:28.655099 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:28.656573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:28.725718 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cb957-7339-4ed2-96f1-af133fc8d54e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:28.730459 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac680069-f3e9-4b68-baa6-d6f6babbc0c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:28.731122 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac680069-f3e9-4b68-baa6-d6f6babbc0c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:28.731407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac680069-f3e9-4b68-baa6-d6f6babbc0c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:28.731695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac680069-f3e9-4b68-baa6-d6f6babbc0c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:28.741676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac680069-f3e9-4b68-baa6-d6f6babbc0c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:28.743162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac680069-f3e9-4b68-baa6-d6f6babbc0c3 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:28.753744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 755/1509] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:32:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:32:29.249957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d760071-2940-4355-9bc0-152568e23a67 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:29.251489 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4d760071-2940-4355-9bc0-152568e23a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:29.264830 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d760071-2940-4355-9bc0-152568e23a67 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:29.269540 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe046150-5f3b-4085-8572-9d933eb95f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:29.273192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe046150-5f3b-4085-8572-9d933eb95f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:29.273192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe046150-5f3b-4085-8572-9d933eb95f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:29.273192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe046150-5f3b-4085-8572-9d933eb95f71 admin admin] PATH_INFO: `/v3/projects/8fa6531224744ca6a263c86e59c9f953` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:29.273192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe046150-5f3b-4085-8572-9d933eb95f71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa6531224744ca6a263c86e59c9f953)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:29.274449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-fe046150-5f3b-4085-8572-9d933eb95f71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:32:29.274449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Jun 07 08:32:29.274449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: Jun 07 08:32:29.275148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe046150-5f3b-4085-8572-9d933eb95f71 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:29.338485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fe046150-5f3b-4085-8572-9d933eb95f71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa6531224744ca6a263c86e59c9f953', 'request_id': 'req-fe046150-5f3b-4085-8572-9d933eb95f71'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:32:29.344423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fe046150-5f3b-4085-8572-9d933eb95f71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa6531224744ca6a263c86e59c9f953', 'request_id': 'req-fe046150-5f3b-4085-8572-9d933eb95f71'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:32:29.354595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fe046150-5f3b-4085-8572-9d933eb95f71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa6531224744ca6a263c86e59c9f953', 'request_id': 'req-fe046150-5f3b-4085-8572-9d933eb95f71'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:32:29.373767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fe046150-5f3b-4085-8572-9d933eb95f71 admin admin] The token cache is being invalidate because project 8fa6531224744ca6a263c86e59c9f953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:32:29.374058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fe046150-5f3b-4085-8572-9d933eb95f71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:32:29.374715 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fe046150-5f3b-4085-8572-9d933eb95f71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:32:29.376364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 755/1510] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:32:29 2022] DELETE /identity/v3/projects/8fa6531224744ca6a263c86e59c9f953 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:32:38.749169 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:32:39.370734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:34:02.996317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069852f-0366-4943-98f9-a6691f615ce2 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:02.996926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069852f-0366-4943-98f9-a6691f615ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:02.996926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069852f-0366-4943-98f9-a6691f615ce2 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:02.998309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:34:03.003084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f68879-be31-4ccd-bd75-1009d77925c5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:03.003568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f68879-be31-4ccd-bd75-1009d77925c5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:03.003675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f68879-be31-4ccd-bd75-1009d77925c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:03.021735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-98f68879-be31-4ccd-bd75-1009d77925c5 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:03.022947 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-98f68879-be31-4ccd-bd75-1009d77925c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:03.083688 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:03.255209 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069852f-0366-4943-98f9-a6691f615ce2 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:03.256895 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0069852f-0366-4943-98f9-a6691f615ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:03.338546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069852f-0366-4943-98f9-a6691f615ce2 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:03.344136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e2009-c94a-4a2c-9731-6647fcdbfb07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:03.344790 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e2009-c94a-4a2c-9731-6647fcdbfb07 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:03.345094 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e2009-c94a-4a2c-9731-6647fcdbfb07 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:03.345387 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e2009-c94a-4a2c-9731-6647fcdbfb07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:03.355056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e2009-c94a-4a2c-9731-6647fcdbfb07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:03.356645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e2009-c94a-4a2c-9731-6647fcdbfb07 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:03.366479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 757/1513] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:34:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:34:04.309361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e525d6f-5c4d-4e15-b511-67bd808fbdf3 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:04.309929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e525d6f-5c4d-4e15-b511-67bd808fbdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:04.309929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e525d6f-5c4d-4e15-b511-67bd808fbdf3 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:04.311541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:34:04.316885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622c31f-fef2-4797-8ea6-9feaa5df17b5 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:04.317505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622c31f-fef2-4797-8ea6-9feaa5df17b5 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:04.317505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622c31f-fef2-4797-8ea6-9feaa5df17b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:04.337150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-a622c31f-fef2-4797-8ea6-9feaa5df17b5 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:04.338387 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a622c31f-fef2-4797-8ea6-9feaa5df17b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:04.347276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:34:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:04.390975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24bc4b-d35e-4d31-8c76-cc5195a8cbf4 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:04.391379 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24bc4b-d35e-4d31-8c76-cc5195a8cbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:04.391725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24bc4b-d35e-4d31-8c76-cc5195a8cbf4 None None] PATH_INFO: `/v3` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:04.393159 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 758/1516] 10.222.0.41 () {54 vars in 925 bytes} [Tue Jun 7 08:34:04 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:34:04.399298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de70be-677c-4b6c-8e32-33e4b47137c6 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:04.399734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de70be-677c-4b6c-8e32-33e4b47137c6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:04.400042 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de70be-677c-4b6c-8e32-33e4b47137c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:04.420834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e7de70be-677c-4b6c-8e32-33e4b47137c6 None None] MFA Rules not processed for user `7a7d1e8a0f4b4511a7dc8b27651744c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:04.422525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e7de70be-677c-4b6c-8e32-33e4b47137c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:04.494602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 759/1517] 10.222.0.41 () {58 vars in 1015 bytes} [Tue Jun 7 08:34:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:04.505387 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24bc4b-d35e-4d31-8c76-cc5195a8cbf4 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:04.521231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24bc4b-d35e-4d31-8c76-cc5195a8cbf4 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:04.527021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625aec91-4857-4421-91cd-38f2313ef79c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:04.527676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625aec91-4857-4421-91cd-38f2313ef79c service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:04.527967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625aec91-4857-4421-91cd-38f2313ef79c service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:04.528320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625aec91-4857-4421-91cd-38f2313ef79c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:04.537181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625aec91-4857-4421-91cd-38f2313ef79c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:04.538733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625aec91-4857-4421-91cd-38f2313ef79c service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:04.548427 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 759/1518] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:34:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:34:08.494931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92081d06-d271-4b28-8aca-6218cb63711a None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:08.495493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92081d06-d271-4b28-8aca-6218cb63711a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:08.495493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92081d06-d271-4b28-8aca-6218cb63711a None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:08.496874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 760/1519] 10.222.0.39 () {54 vars in 960 bytes} [Tue Jun 7 08:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:34:08.501915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6d88f-418a-42e0-81f1-74f996d4fb89 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:08.502352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6d88f-418a-42e0-81f1-74f996d4fb89 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:08.502595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6d88f-418a-42e0-81f1-74f996d4fb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:08.520587 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-29e6d88f-418a-42e0-81f1-74f996d4fb89 None None] MFA Rules not processed for user `7a7d1e8a0f4b4511a7dc8b27651744c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:08.521835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-29e6d88f-418a-42e0-81f1-74f996d4fb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:08.531414 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 760/1520] 10.222.0.39 () {58 vars in 1057 bytes} [Tue Jun 7 08:34:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:08.539064 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92081d06-d271-4b28-8aca-6218cb63711a None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:08.553295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92081d06-d271-4b28-8aca-6218cb63711a None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:08.558332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c849be-5f92-4aa7-af9c-164eed1528af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:08.559001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c849be-5f92-4aa7-af9c-164eed1528af service placement] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:08.559289 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c849be-5f92-4aa7-af9c-164eed1528af service placement] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:08.559568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c849be-5f92-4aa7-af9c-164eed1528af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:08.567972 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c849be-5f92-4aa7-af9c-164eed1528af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:08.569591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c849be-5f92-4aa7-af9c-164eed1528af service placement] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:08.580475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 761/1521] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:34:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:34:15.898585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff078cc0-c213-4195-883a-b03619caf420 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:15.899383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff078cc0-c213-4195-883a-b03619caf420 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:15.899383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff078cc0-c213-4195-883a-b03619caf420 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:15.901055 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:34:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:34:15.907410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ecaeb-a354-4b12-9a38-535f12d911ef None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:15.908445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ecaeb-a354-4b12-9a38-535f12d911ef None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:15.908445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ecaeb-a354-4b12-9a38-535f12d911ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:15.932661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-ad9ecaeb-a354-4b12-9a38-535f12d911ef None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:15.933919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ad9ecaeb-a354-4b12-9a38-535f12d911ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:15.942253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:16.097926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff078cc0-c213-4195-883a-b03619caf420 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:16.116106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff078cc0-c213-4195-883a-b03619caf420 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:16.123149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff3532-5216-4954-afc6-907e0a9c5e64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:16.123820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff3532-5216-4954-afc6-907e0a9c5e64 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:16.124109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff3532-5216-4954-afc6-907e0a9c5e64 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:16.124390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff3532-5216-4954-afc6-907e0a9c5e64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:16.140057 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff3532-5216-4954-afc6-907e0a9c5e64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:16.141909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff3532-5216-4954-afc6-907e0a9c5e64 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:16.155119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 762/1524] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:34:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:34:17.062058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6b8e3-d29e-4682-b6bf-19bf897f5ed7 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:17.062639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6b8e3-d29e-4682-b6bf-19bf897f5ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:17.063019 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6b8e3-d29e-4682-b6bf-19bf897f5ed7 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:17.064961 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:34:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:34:17.070963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8765c2d-5352-4c7d-91f1-d3ca3437567c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:17.071770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8765c2d-5352-4c7d-91f1-d3ca3437567c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:17.071770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8765c2d-5352-4c7d-91f1-d3ca3437567c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:17.094106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a8765c2d-5352-4c7d-91f1-d3ca3437567c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:17.095354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a8765c2d-5352-4c7d-91f1-d3ca3437567c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:17.104195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:17.261776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6b8e3-d29e-4682-b6bf-19bf897f5ed7 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:17.277359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6b8e3-d29e-4682-b6bf-19bf897f5ed7 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:17.283805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762560d1-7d37-4c12-9e08-f18a6d7c1300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:17.284791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762560d1-7d37-4c12-9e08-f18a6d7c1300 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:17.285254 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762560d1-7d37-4c12-9e08-f18a6d7c1300 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:17.285681 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762560d1-7d37-4c12-9e08-f18a6d7c1300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:17.299650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762560d1-7d37-4c12-9e08-f18a6d7c1300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:17.302388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762560d1-7d37-4c12-9e08-f18a6d7c1300 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:17.313233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 764/1527] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:34:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:34:21.526650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d2e39-4488-4d13-b194-b1d982025c0a None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:21.527201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d2e39-4488-4d13-b194-b1d982025c0a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:21.527201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d2e39-4488-4d13-b194-b1d982025c0a None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:21.528595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:34:21.533822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb40c2-b6dc-4e73-94af-5e17dcd32107 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:21.534260 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb40c2-b6dc-4e73-94af-5e17dcd32107 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:21.534382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb40c2-b6dc-4e73-94af-5e17dcd32107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:21.552271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-f4eb40c2-b6dc-4e73-94af-5e17dcd32107 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:21.553506 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f4eb40c2-b6dc-4e73-94af-5e17dcd32107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:21.561546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:21.599046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d2e39-4488-4d13-b194-b1d982025c0a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:21.600715 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d08d2e39-4488-4d13-b194-b1d982025c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:21.657906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d2e39-4488-4d13-b194-b1d982025c0a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:21.661496 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5049a68-691e-4efe-abe1-eeec13b87bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:21.662151 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5049a68-691e-4efe-abe1-eeec13b87bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:21.662463 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5049a68-691e-4efe-abe1-eeec13b87bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:21.662744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5049a68-691e-4efe-abe1-eeec13b87bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:21.671849 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5049a68-691e-4efe-abe1-eeec13b87bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:21.673294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5049a68-691e-4efe-abe1-eeec13b87bd2 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:21.682659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 765/1530] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:34:23.996726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046c076-25ee-4168-a1b9-df7ebcf64a75 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:23.997316 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046c076-25ee-4168-a1b9-df7ebcf64a75 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:23.997316 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046c076-25ee-4168-a1b9-df7ebcf64a75 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:23.998705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:34:24.004141 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66371a6f-ac28-4cda-88a5-e8f428bc5616 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:24.004630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66371a6f-ac28-4cda-88a5-e8f428bc5616 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:24.004747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66371a6f-ac28-4cda-88a5-e8f428bc5616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:24.024700 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-66371a6f-ac28-4cda-88a5-e8f428bc5616 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:24.025993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-66371a6f-ac28-4cda-88a5-e8f428bc5616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:24.033790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:24.069781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046c076-25ee-4168-a1b9-df7ebcf64a75 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:24.082683 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046c076-25ee-4168-a1b9-df7ebcf64a75 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:24.086455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8540f5-6575-42dc-8475-3647158375e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:24.087099 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8540f5-6575-42dc-8475-3647158375e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:24.087386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8540f5-6575-42dc-8475-3647158375e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:24.087664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8540f5-6575-42dc-8475-3647158375e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:24.096869 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8540f5-6575-42dc-8475-3647158375e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:24.098354 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8540f5-6575-42dc-8475-3647158375e8 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:24.108017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 767/1533] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:34:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:34:27.893551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702d4b0-3526-46b5-a46c-b17211226bc7 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:27.894148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702d4b0-3526-46b5-a46c-b17211226bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:27.894148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702d4b0-3526-46b5-a46c-b17211226bc7 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:27.895514 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:34:27.902503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ec3a3-eb75-48aa-9143-a54e09f7ca88 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:27.903024 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ec3a3-eb75-48aa-9143-a54e09f7ca88 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:27.904154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ec3a3-eb75-48aa-9143-a54e09f7ca88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:27.925335 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-703ec3a3-eb75-48aa-9143-a54e09f7ca88 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:27.926560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-703ec3a3-eb75-48aa-9143-a54e09f7ca88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:27.934748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:28.209774 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702d4b0-3526-46b5-a46c-b17211226bc7 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:28.211464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b702d4b0-3526-46b5-a46c-b17211226bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:28.270189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702d4b0-3526-46b5-a46c-b17211226bc7 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:28.274096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1b5df-afd4-4b43-9f52-3a76df131c1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:28.274798 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1b5df-afd4-4b43-9f52-3a76df131c1a service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:28.275081 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1b5df-afd4-4b43-9f52-3a76df131c1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:28.275358 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1b5df-afd4-4b43-9f52-3a76df131c1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:28.284493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1b5df-afd4-4b43-9f52-3a76df131c1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:28.285945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1b5df-afd4-4b43-9f52-3a76df131c1a service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:28.291722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:34:28.297136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 768/1536] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:34:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:34:30.581568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9de4a0-a3e5-4570-bfb2-392a4f1aa00c None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:30.582449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9de4a0-a3e5-4570-bfb2-392a4f1aa00c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:30.582567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9de4a0-a3e5-4570-bfb2-392a4f1aa00c None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:30.584064 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:34:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:34:30.590053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a49d42-a071-4ca3-a5a7-c849970dea38 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:30.590465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a49d42-a071-4ca3-a5a7-c849970dea38 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:30.590607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a49d42-a071-4ca3-a5a7-c849970dea38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:30.608318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-47a49d42-a071-4ca3-a5a7-c849970dea38 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:30.609531 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-47a49d42-a071-4ca3-a5a7-c849970dea38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:30.617419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:30.887942 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9de4a0-a3e5-4570-bfb2-392a4f1aa00c None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:30.889637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0d9de4a0-a3e5-4570-bfb2-392a4f1aa00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:30.897452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=94 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:34:30.904908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9de4a0-a3e5-4570-bfb2-392a4f1aa00c None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:30.908586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534f29f-be94-46b9-a3ca-7b430ae50a09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:30.909295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534f29f-be94-46b9-a3ca-7b430ae50a09 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:30.909581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534f29f-be94-46b9-a3ca-7b430ae50a09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:30.909861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534f29f-be94-46b9-a3ca-7b430ae50a09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:30.920164 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534f29f-be94-46b9-a3ca-7b430ae50a09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:30.921629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534f29f-be94-46b9-a3ca-7b430ae50a09 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:30.931497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 770/1539] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:34:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:34:40.603168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:34:40.926789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:03.982898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f4f3f-3cca-4727-a5b0-464f3fc4eedc None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:03.983567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f4f3f-3cca-4727-a5b0-464f3fc4eedc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:03.983567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f4f3f-3cca-4727-a5b0-464f3fc4eedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.010189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5f8f4f3f-3cca-4727-a5b0-464f3fc4eedc None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.011483 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5f8f4f3f-3cca-4727-a5b0-464f3fc4eedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.023680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fb4c9-15c9-46d6-9b18-9b1adb22a22e None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.024604 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fb4c9-15c9-46d6-9b18-9b1adb22a22e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.024604 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fb4c9-15c9-46d6-9b18-9b1adb22a22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.026789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 770/1540] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:04.039082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5c2fc-16c0-4271-9dd2-5c7a89729739 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.039493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5c2fc-16c0-4271-9dd2-5c7a89729739 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.039757 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5c2fc-16c0-4271-9dd2-5c7a89729739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.062540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d76fb4c9-15c9-46d6-9b18-9b1adb22a22e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.063915 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d76fb4c9-15c9-46d6-9b18-9b1adb22a22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.081159 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7fb5c2fc-16c0-4271-9dd2-5c7a89729739 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.081159 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7fb5c2fc-16c0-4271-9dd2-5c7a89729739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.081481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 771/1541] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:04.090178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8030f-d991-436a-a163-8dfab3c57069 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.090460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8030f-d991-436a-a163-8dfab3c57069 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.090972 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8030f-d991-436a-a163-8dfab3c57069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.099124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 771/1542] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:04.106424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5c2fc-16c0-4271-9dd2-5c7a89729739 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.119277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-cbf8030f-d991-436a-a163-8dfab3c57069 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.125106 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-cbf8030f-d991-436a-a163-8dfab3c57069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.129538 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5c2fc-16c0-4271-9dd2-5c7a89729739 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.141924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 772/1543] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:04.142967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dacf9-c259-4fba-906e-d19b7af25dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.143744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dacf9-c259-4fba-906e-d19b7af25dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.144037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dacf9-c259-4fba-906e-d19b7af25dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.144315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dacf9-c259-4fba-906e-d19b7af25dc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.145102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73dacf9-c259-4fba-906e-d19b7af25dc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:04.145376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73dacf9-c259-4fba-906e-d19b7af25dc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.147053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73dacf9-c259-4fba-906e-d19b7af25dc4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.149465 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8030f-d991-436a-a163-8dfab3c57069 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.159312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8030f-d991-436a-a163-8dfab3c57069 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.169286 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 772/1544] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:36:04.171480 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad4b39-68b2-4484-83f6-78ff34c1f8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.172381 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad4b39-68b2-4484-83f6-78ff34c1f8ee admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.172860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad4b39-68b2-4484-83f6-78ff34c1f8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.173408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad4b39-68b2-4484-83f6-78ff34c1f8ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.174330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad4b39-68b2-4484-83f6-78ff34c1f8ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:04.174831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad4b39-68b2-4484-83f6-78ff34c1f8ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.177521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dacf9-c259-4fba-906e-d19b7af25dc4 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.177744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: WARNING py.warnings [None req-53ad4b39-68b2-4484-83f6-78ff34c1f8ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:36:04.177744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: warnings.warn(msg) Jun 07 08:36:04.177744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: Jun 07 08:36:04.179174 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad4b39-68b2-4484-83f6-78ff34c1f8ee admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.193612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dacf9-c259-4fba-906e-d19b7af25dc4 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.203376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1dcdb-ea27-4bfc-81c6-54c9d5ec41b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.203805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 773/1545] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:36:04.204196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1dcdb-ea27-4bfc-81c6-54c9d5ec41b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.204485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1dcdb-ea27-4bfc-81c6-54c9d5ec41b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.204822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1dcdb-ea27-4bfc-81c6-54c9d5ec41b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.205541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1dcdb-ea27-4bfc-81c6-54c9d5ec41b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.206561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-ebe1dcdb-ea27-4bfc-81c6-54c9d5ec41b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:36:04.206561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Jun 07 08:36:04.206561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: Jun 07 08:36:04.208110 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1dcdb-ea27-4bfc-81c6-54c9d5ec41b1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.209250 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad4b39-68b2-4484-83f6-78ff34c1f8ee admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.223108 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad4b39-68b2-4484-83f6-78ff34c1f8ee admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.235373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf57882-74e0-46f4-a248-537f820cc816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.236401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf57882-74e0-46f4-a248-537f820cc816 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.236963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf57882-74e0-46f4-a248-537f820cc816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.237457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf57882-74e0-46f4-a248-537f820cc816 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.238401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf57882-74e0-46f4-a248-537f820cc816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.240624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf57882-74e0-46f4-a248-537f820cc816 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.255897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 773/1546] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/projects => generated 393 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:36:04.260311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1dcdb-ea27-4bfc-81c6-54c9d5ec41b1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.303185 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 774/1547] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:36:04.313487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf57882-74e0-46f4-a248-537f820cc816 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.370156 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1dcdb-ea27-4bfc-81c6-54c9d5ec41b1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.430346 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf57882-74e0-46f4-a248-537f820cc816 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.438825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de279a-987f-4e4c-8e30-edcc7d96b3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.439504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de279a-987f-4e4c-8e30-edcc7d96b3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.439790 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de279a-987f-4e4c-8e30-edcc7d96b3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.440068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de279a-987f-4e4c-8e30-edcc7d96b3a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.440765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de279a-987f-4e4c-8e30-edcc7d96b3a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.442151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: WARNING py.warnings [None req-51de279a-987f-4e4c-8e30-edcc7d96b3a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:36:04.442151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: warnings.warn(msg) Jun 07 08:36:04.442151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: Jun 07 08:36:04.444685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de279a-987f-4e4c-8e30-edcc7d96b3a0 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.450785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7761d17-21b5-43b7-adf3-2a2e7e27a8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.452360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7761d17-21b5-43b7-adf3-2a2e7e27a8ff admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.453660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7761d17-21b5-43b7-adf3-2a2e7e27a8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.454734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7761d17-21b5-43b7-adf3-2a2e7e27a8ff admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.456373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7761d17-21b5-43b7-adf3-2a2e7e27a8ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.458875 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7761d17-21b5-43b7-adf3-2a2e7e27a8ff admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.470816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 775/1548] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:04.472965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d40ec5-a28c-4dd5-951c-5a3e35c0ffd7 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.473636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d40ec5-a28c-4dd5-951c-5a3e35c0ffd7 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.474314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d40ec5-a28c-4dd5-951c-5a3e35c0ffd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.487144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 774/1549] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/users => generated 297 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:36:04.491452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08311b16-2e2e-404b-bca9-86e31586fbe9 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.491452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08311b16-2e2e-404b-bca9-86e31586fbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.491452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08311b16-2e2e-404b-bca9-86e31586fbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.504402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e5d40ec5-a28c-4dd5-951c-5a3e35c0ffd7 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.505685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e5d40ec5-a28c-4dd5-951c-5a3e35c0ffd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.513952 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-08311b16-2e2e-404b-bca9-86e31586fbe9 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.515318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-08311b16-2e2e-404b-bca9-86e31586fbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.515443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 776/1550] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:04.516456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618d6c1-6ed5-41aa-b0ff-05b6490dd165 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.516751 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618d6c1-6ed5-41aa-b0ff-05b6490dd165 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.517026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618d6c1-6ed5-41aa-b0ff-05b6490dd165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.524331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 775/1551] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:04.525703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c7c21-ba5f-4cbc-96d3-1864f6556793 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.526001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c7c21-ba5f-4cbc-96d3-1864f6556793 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.526281 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c7c21-ba5f-4cbc-96d3-1864f6556793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.535408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1618d6c1-6ed5-41aa-b0ff-05b6490dd165 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.536610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1618d6c1-6ed5-41aa-b0ff-05b6490dd165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.545228 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-060c7c21-ba5f-4cbc-96d3-1864f6556793 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.545411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 777/1552] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:04.545884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618d6c1-6ed5-41aa-b0ff-05b6490dd165 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.546412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-060c7c21-ba5f-4cbc-96d3-1864f6556793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.556254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 776/1553] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:04.557108 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c7c21-ba5f-4cbc-96d3-1864f6556793 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.558991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618d6c1-6ed5-41aa-b0ff-05b6490dd165 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.563982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de8438-697b-4944-b214-e580d5bfc947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.564636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de8438-697b-4944-b214-e580d5bfc947 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.564918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de8438-697b-4944-b214-e580d5bfc947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.565227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de8438-697b-4944-b214-e580d5bfc947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.565970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de8438-697b-4944-b214-e580d5bfc947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.567315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c7c21-ba5f-4cbc-96d3-1864f6556793 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.567572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de8438-697b-4944-b214-e580d5bfc947 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.573661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4754516-c398-4731-9585-065bfe1ba25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.574025 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4754516-c398-4731-9585-065bfe1ba25c admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.574314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4754516-c398-4731-9585-065bfe1ba25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.574609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4754516-c398-4731-9585-065bfe1ba25c admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.575321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4754516-c398-4731-9585-065bfe1ba25c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.576922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4754516-c398-4731-9585-065bfe1ba25c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.578246 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 778/1554] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:04.579488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c0210-146d-48fc-86b3-304e0de6e1c6 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.579779 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c0210-146d-48fc-86b3-304e0de6e1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.580060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c0210-146d-48fc-86b3-304e0de6e1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.590116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 777/1555] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:04.591599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bc9e0-1f7e-4f9f-b983-0e21f0e3479d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.592060 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bc9e0-1f7e-4f9f-b983-0e21f0e3479d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.592506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bc9e0-1f7e-4f9f-b983-0e21f0e3479d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.598892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-338c0210-146d-48fc-86b3-304e0de6e1c6 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.600237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-338c0210-146d-48fc-86b3-304e0de6e1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.608986 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 779/1556] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:04.613271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5003b-97ef-4324-bdee-713a005da572 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.613271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5003b-97ef-4324-bdee-713a005da572 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.613271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5003b-97ef-4324-bdee-713a005da572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.613562 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-975bc9e0-1f7e-4f9f-b983-0e21f0e3479d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.613562 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-975bc9e0-1f7e-4f9f-b983-0e21f0e3479d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.622069 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 778/1557] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:04.623441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacad61-cc26-410a-942d-1807b6341b8b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.623726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacad61-cc26-410a-942d-1807b6341b8b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.624004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacad61-cc26-410a-942d-1807b6341b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.637475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-afa5003b-97ef-4324-bdee-713a005da572 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.638672 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-afa5003b-97ef-4324-bdee-713a005da572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.642287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7eacad61-cc26-410a-942d-1807b6341b8b None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.643452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7eacad61-cc26-410a-942d-1807b6341b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.646526 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 780/1558] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:04.651103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96870cc-efe9-4c34-89b6-f5f6291f048f None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.651798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96870cc-efe9-4c34-89b6-f5f6291f048f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.652216 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96870cc-efe9-4c34-89b6-f5f6291f048f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.652733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 779/1559] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:04.658841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e784c-409b-458b-b8a7-4457cc834076 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.658841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e784c-409b-458b-b8a7-4457cc834076 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.658841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e784c-409b-458b-b8a7-4457cc834076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.675315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d68e784c-409b-458b-b8a7-4457cc834076 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.675890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-a96870cc-efe9-4c34-89b6-f5f6291f048f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.676511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d68e784c-409b-458b-b8a7-4457cc834076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.677067 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a96870cc-efe9-4c34-89b6-f5f6291f048f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.685059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 780/1560] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:04.685815 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e784c-409b-458b-b8a7-4457cc834076 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.686303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 781/1561] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:04.689939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96870cc-efe9-4c34-89b6-f5f6291f048f None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.693789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:04.699396 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e784c-409b-458b-b8a7-4457cc834076 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.700084 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96870cc-efe9-4c34-89b6-f5f6291f048f None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.705573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65210500-a887-48bf-b7f7-474ebd7c1f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.706358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65210500-a887-48bf-b7f7-474ebd7c1f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.706740 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65210500-a887-48bf-b7f7-474ebd7c1f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.706881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35a30b-2306-45ef-b852-80644def198e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.707383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65210500-a887-48bf-b7f7-474ebd7c1f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.707887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35a30b-2306-45ef-b852-80644def198e admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.708244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35a30b-2306-45ef-b852-80644def198e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.708773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65210500-a887-48bf-b7f7-474ebd7c1f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.709163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35a30b-2306-45ef-b852-80644def198e admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.710126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35a30b-2306-45ef-b852-80644def198e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.710568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65210500-a887-48bf-b7f7-474ebd7c1f3d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.711984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35a30b-2306-45ef-b852-80644def198e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.721132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 782/1562] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:04.722517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b300cf2-a489-482a-a75d-588ce4683550 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.722829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 781/1563] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:04.724104 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca17a5-81ea-4e75-a92c-0f1a9c27ad19 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.724387 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca17a5-81ea-4e75-a92c-0f1a9c27ad19 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.724669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca17a5-81ea-4e75-a92c-0f1a9c27ad19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.726802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b300cf2-a489-482a-a75d-588ce4683550 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.727091 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b300cf2-a489-482a-a75d-588ce4683550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.746986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-64ca17a5-81ea-4e75-a92c-0f1a9c27ad19 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.748213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-64ca17a5-81ea-4e75-a92c-0f1a9c27ad19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.748610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1b300cf2-a489-482a-a75d-588ce4683550 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:04.749807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1b300cf2-a489-482a-a75d-588ce4683550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:04.756450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 782/1564] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:04.757935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 783/1565] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:04.758767 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b300cf2-a489-482a-a75d-588ce4683550 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.760880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca17a5-81ea-4e75-a92c-0f1a9c27ad19 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.770712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca17a5-81ea-4e75-a92c-0f1a9c27ad19 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.770936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b300cf2-a489-482a-a75d-588ce4683550 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.775364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac76d2-8cf8-45c3-a713-14992b945689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.775503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc9b18-01c8-438c-93ff-b7b389ede63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.776024 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac76d2-8cf8-45c3-a713-14992b945689 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.776135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc9b18-01c8-438c-93ff-b7b389ede63f admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.776308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac76d2-8cf8-45c3-a713-14992b945689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.776412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc9b18-01c8-438c-93ff-b7b389ede63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.776581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc9b18-01c8-438c-93ff-b7b389ede63f admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.776740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac76d2-8cf8-45c3-a713-14992b945689 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.777329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac76d2-8cf8-45c3-a713-14992b945689 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:04.777445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc9b18-01c8-438c-93ff-b7b389ede63f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:04.777636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac76d2-8cf8-45c3-a713-14992b945689 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.777734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc9b18-01c8-438c-93ff-b7b389ede63f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.779229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac76d2-8cf8-45c3-a713-14992b945689 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.779375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc9b18-01c8-438c-93ff-b7b389ede63f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.791639 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 783/1566] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:04.791921 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 784/1567] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:04.792763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc9b18-01c8-438c-93ff-b7b389ede63f admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.799803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac76d2-8cf8-45c3-a713-14992b945689 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.804293 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc9b18-01c8-438c-93ff-b7b389ede63f admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.809285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835361d9-3462-4893-b989-b58cd1d9154d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.809946 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835361d9-3462-4893-b989-b58cd1d9154d admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.810155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac76d2-8cf8-45c3-a713-14992b945689 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.810280 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835361d9-3462-4893-b989-b58cd1d9154d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.810496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835361d9-3462-4893-b989-b58cd1d9154d admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.811218 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835361d9-3462-4893-b989-b58cd1d9154d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:04.811523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835361d9-3462-4893-b989-b58cd1d9154d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.813149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835361d9-3462-4893-b989-b58cd1d9154d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.814690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ffc98-ffba-4658-aa72-dcffb725a884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.815400 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ffc98-ffba-4658-aa72-dcffb725a884 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.815621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ffc98-ffba-4658-aa72-dcffb725a884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.815894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ffc98-ffba-4658-aa72-dcffb725a884 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.816604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ffc98-ffba-4658-aa72-dcffb725a884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:04.816907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ffc98-ffba-4658-aa72-dcffb725a884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.818505 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ffc98-ffba-4658-aa72-dcffb725a884 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.825568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 785/1568] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:04.826367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835361d9-3462-4893-b989-b58cd1d9154d admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.838956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835361d9-3462-4893-b989-b58cd1d9154d admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.840017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 784/1569] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:04.840929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ffc98-ffba-4658-aa72-dcffb725a884 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.843932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348c13e-36fe-4612-8582-54f4b4985cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.844809 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348c13e-36fe-4612-8582-54f4b4985cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.845120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348c13e-36fe-4612-8582-54f4b4985cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.845401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348c13e-36fe-4612-8582-54f4b4985cf9 admin admin] PATH_INFO: `/v3/projects/e7bbcea61e7549fbac7aecd75f14c316/users/3e6fe6fa3fc14cca97064b49403eb68b/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.852233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ffc98-ffba-4658-aa72-dcffb725a884 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.857313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48cead-fdf7-40ea-bde8-ea785214cb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.858196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48cead-fdf7-40ea-bde8-ea785214cb36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.858335 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:04.858478 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48cead-fdf7-40ea-bde8-ea785214cb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.858749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48cead-fdf7-40ea-bde8-ea785214cb36 admin admin] PATH_INFO: `/v3/projects/973dc13b7788475bbe4cff5dac7b873e/users/e61d179002a447d89160d8cb435089eb/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.864273 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348c13e-36fe-4612-8582-54f4b4985cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bbcea61e7549fbac7aecd75f14c316, user_id=3e6fe6fa3fc14cca97064b49403eb68b, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.867521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348c13e-36fe-4612-8582-54f4b4985cf9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.874680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48cead-fdf7-40ea-bde8-ea785214cb36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=973dc13b7788475bbe4cff5dac7b873e, user_id=e61d179002a447d89160d8cb435089eb, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.875797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-ca48cead-fdf7-40ea-bde8-ea785214cb36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:36:04.875797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Jun 07 08:36:04.875797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: Jun 07 08:36:04.877818 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48cead-fdf7-40ea-bde8-ea785214cb36 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.881675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 786/1570] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:04 2022] PUT /identity/v3/projects/e7bbcea61e7549fbac7aecd75f14c316/users/3e6fe6fa3fc14cca97064b49403eb68b/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:04.882610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348c13e-36fe-4612-8582-54f4b4985cf9 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.894269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 785/1571] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:04 2022] PUT /identity/v3/projects/973dc13b7788475bbe4cff5dac7b873e/users/e61d179002a447d89160d8cb435089eb/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:04.895465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48cead-fdf7-40ea-bde8-ea785214cb36 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.960757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348c13e-36fe-4612-8582-54f4b4985cf9 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.962103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48cead-fdf7-40ea-bde8-ea785214cb36 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.965353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569f7e6-423f-4557-aa57-d50b99a37abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.966015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569f7e6-423f-4557-aa57-d50b99a37abd admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.966298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569f7e6-423f-4557-aa57-d50b99a37abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.966575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569f7e6-423f-4557-aa57-d50b99a37abd admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.966683 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f75b5-0f58-4c6c-a9aa-2ed71f0c043a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:04.967288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f75b5-0f58-4c6c-a9aa-2ed71f0c043a admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:04.967388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569f7e6-423f-4557-aa57-d50b99a37abd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:04.967577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f75b5-0f58-4c6c-a9aa-2ed71f0c043a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:04.967688 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569f7e6-423f-4557-aa57-d50b99a37abd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.967856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f75b5-0f58-4c6c-a9aa-2ed71f0c043a admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:04.968578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f75b5-0f58-4c6c-a9aa-2ed71f0c043a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:04.968907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f75b5-0f58-4c6c-a9aa-2ed71f0c043a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:04.969204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569f7e6-423f-4557-aa57-d50b99a37abd admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.970513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f75b5-0f58-4c6c-a9aa-2ed71f0c043a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:04.987484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 787/1572] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:04.987484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569f7e6-423f-4557-aa57-d50b99a37abd admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.987795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 786/1573] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:04.987795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f75b5-0f58-4c6c-a9aa-2ed71f0c043a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:04.998402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569f7e6-423f-4557-aa57-d50b99a37abd admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:04.999643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f75b5-0f58-4c6c-a9aa-2ed71f0c043a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.004160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8d798-5133-4170-804c-60ab46105b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.004669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e9c57-fcc6-4000-8e75-e2092a7b9d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.004976 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8d798-5133-4170-804c-60ab46105b4a admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.005366 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8d798-5133-4170-804c-60ab46105b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.005558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e9c57-fcc6-4000-8e75-e2092a7b9d26 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.005791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8d798-5133-4170-804c-60ab46105b4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.005955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e9c57-fcc6-4000-8e75-e2092a7b9d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.007350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8d798-5133-4170-804c-60ab46105b4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.007635 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e9c57-fcc6-4000-8e75-e2092a7b9d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.008839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e9c57-fcc6-4000-8e75-e2092a7b9d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.011276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8d798-5133-4170-804c-60ab46105b4a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.012915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e9c57-fcc6-4000-8e75-e2092a7b9d26 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.045361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 787/1574] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:05.046013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e9c57-fcc6-4000-8e75-e2092a7b9d26 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.053252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 788/1575] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:04 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:05.054929 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8d798-5133-4170-804c-60ab46105b4a admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.118070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e9c57-fcc6-4000-8e75-e2092a7b9d26 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.122721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb427c1-66a8-41f4-9cd4-97d92cecafdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.123383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb427c1-66a8-41f4-9cd4-97d92cecafdd admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.123666 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb427c1-66a8-41f4-9cd4-97d92cecafdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.123942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb427c1-66a8-41f4-9cd4-97d92cecafdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.124631 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb427c1-66a8-41f4-9cd4-97d92cecafdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.124736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8d798-5133-4170-804c-60ab46105b4a admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.126584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb427c1-66a8-41f4-9cd4-97d92cecafdd admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.129238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a536097-5fcf-4250-b7f9-1dca69dd97d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.129890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a536097-5fcf-4250-b7f9-1dca69dd97d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.130169 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a536097-5fcf-4250-b7f9-1dca69dd97d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.130713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a536097-5fcf-4250-b7f9-1dca69dd97d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.131117 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a536097-5fcf-4250-b7f9-1dca69dd97d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.133756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a536097-5fcf-4250-b7f9-1dca69dd97d3 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.140858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:05.156009 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 788/1576] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:05 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:05.160559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa452843-4f47-44fa-a5fd-660df548bc3c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.160858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa452843-4f47-44fa-a5fd-660df548bc3c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.161050 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 789/1577] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:05 2022] POST /identity/v3/projects => generated 405 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:05.161412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa452843-4f47-44fa-a5fd-660df548bc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.165371 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6552fa-d550-4db8-94b7-9f28340ea3f6 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.165662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6552fa-d550-4db8-94b7-9f28340ea3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.165941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6552fa-d550-4db8-94b7-9f28340ea3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.180428 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-fa452843-4f47-44fa-a5fd-660df548bc3c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:05.181732 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fa452843-4f47-44fa-a5fd-660df548bc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:05.184437 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-2d6552fa-d550-4db8-94b7-9f28340ea3f6 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:05.185660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-2d6552fa-d550-4db8-94b7-9f28340ea3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:05.230494 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:05.246311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 789/1578] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:05.247327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa452843-4f47-44fa-a5fd-660df548bc3c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.251674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 790/1579] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:05.252707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6552fa-d550-4db8-94b7-9f28340ea3f6 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.260556 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa452843-4f47-44fa-a5fd-660df548bc3c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.262405 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6552fa-d550-4db8-94b7-9f28340ea3f6 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.265237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d376e-f558-49e3-854f-74f6b97bbe3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.265899 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d376e-f558-49e3-854f-74f6b97bbe3f admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.266180 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d376e-f558-49e3-854f-74f6b97bbe3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.266455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d376e-f558-49e3-854f-74f6b97bbe3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.266978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2a6cc-c824-4ebc-947a-27a52748bb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.267140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d376e-f558-49e3-854f-74f6b97bbe3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.267634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2a6cc-c824-4ebc-947a-27a52748bb39 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.267924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2a6cc-c824-4ebc-947a-27a52748bb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.268202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2a6cc-c824-4ebc-947a-27a52748bb39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.268919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2a6cc-c824-4ebc-947a-27a52748bb39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.269070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d376e-f558-49e3-854f-74f6b97bbe3f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.270847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2a6cc-c824-4ebc-947a-27a52748bb39 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.295404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 790/1580] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:05 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:05.296208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d376e-f558-49e3-854f-74f6b97bbe3f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.300985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 791/1581] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:05 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:05.305264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2a6cc-c824-4ebc-947a-27a52748bb39 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.369706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d376e-f558-49e3-854f-74f6b97bbe3f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.386609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2a6cc-c824-4ebc-947a-27a52748bb39 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.392504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59acce-b65b-459e-aad6-049b77500632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.393224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59acce-b65b-459e-aad6-049b77500632 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.393513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59acce-b65b-459e-aad6-049b77500632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.393793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59acce-b65b-459e-aad6-049b77500632 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.394495 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59acce-b65b-459e-aad6-049b77500632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.396436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59acce-b65b-459e-aad6-049b77500632 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.419899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 792/1582] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:05 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:05.420973 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59acce-b65b-459e-aad6-049b77500632 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.431359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59acce-b65b-459e-aad6-049b77500632 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.433884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c220740-6c59-4797-b523-c8d0cb2e6adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.434553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c220740-6c59-4797-b523-c8d0cb2e6adb admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.434834 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c220740-6c59-4797-b523-c8d0cb2e6adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.435107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c220740-6c59-4797-b523-c8d0cb2e6adb admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.435788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c220740-6c59-4797-b523-c8d0cb2e6adb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.436407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d62e2e-7b58-43eb-bd9f-6b497d46c6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.437063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62e2e-7b58-43eb-bd9f-6b497d46c6ab admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.437383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62e2e-7b58-43eb-bd9f-6b497d46c6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.437666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62e2e-7b58-43eb-bd9f-6b497d46c6ab admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.437770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c220740-6c59-4797-b523-c8d0cb2e6adb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.438355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d62e2e-7b58-43eb-bd9f-6b497d46c6ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.440263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d62e2e-7b58-43eb-bd9f-6b497d46c6ab admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.464931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 791/1583] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:05 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:05.464931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c220740-6c59-4797-b523-c8d0cb2e6adb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.467077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 793/1584] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:05 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:05.467978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d62e2e-7b58-43eb-bd9f-6b497d46c6ab admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.473062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c220740-6c59-4797-b523-c8d0cb2e6adb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.478123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ad490-3698-4685-bd9f-e58c0a4f886c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.478792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ad490-3698-4685-bd9f-e58c0a4f886c admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.479076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ad490-3698-4685-bd9f-e58c0a4f886c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.479354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ad490-3698-4685-bd9f-e58c0a4f886c admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.480040 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ad490-3698-4685-bd9f-e58c0a4f886c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.482055 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ad490-3698-4685-bd9f-e58c0a4f886c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.489751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:05.508492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 792/1585] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:05 2022] POST /identity/v3/users => generated 303 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:05.508492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ad490-3698-4685-bd9f-e58c0a4f886c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.509901 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b08ad490-3698-4685-bd9f-e58c0a4f886c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:05.527246 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d62e2e-7b58-43eb-bd9f-6b497d46c6ab admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.531484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b8479-5b0c-4ea0-b807-87c171ed4783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.532135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b8479-5b0c-4ea0-b807-87c171ed4783 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.532420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b8479-5b0c-4ea0-b807-87c171ed4783 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.532690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b8479-5b0c-4ea0-b807-87c171ed4783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.542780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b8479-5b0c-4ea0-b807-87c171ed4783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.544282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b8479-5b0c-4ea0-b807-87c171ed4783 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.554681 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 794/1586] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:05.555549 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b8479-5b0c-4ea0-b807-87c171ed4783 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.564772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b8479-5b0c-4ea0-b807-87c171ed4783 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.569674 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ad490-3698-4685-bd9f-e58c0a4f886c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.569839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea73c8-0ca3-42ca-8d6b-94084cfc33ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.570374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea73c8-0ca3-42ca-8d6b-94084cfc33ee admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.570652 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea73c8-0ca3-42ca-8d6b-94084cfc33ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.570932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea73c8-0ca3-42ca-8d6b-94084cfc33ee admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.571620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea73c8-0ca3-42ca-8d6b-94084cfc33ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.573566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea73c8-0ca3-42ca-8d6b-94084cfc33ee admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.574994 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35636a-066d-44aa-8ffc-4c3644145298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.575821 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35636a-066d-44aa-8ffc-4c3644145298 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.576259 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35636a-066d-44aa-8ffc-4c3644145298 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.576827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35636a-066d-44aa-8ffc-4c3644145298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.594358 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35636a-066d-44aa-8ffc-4c3644145298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.596963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35636a-066d-44aa-8ffc-4c3644145298 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.608121 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 795/1587] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:05 2022] POST /identity/v3/users => generated 291 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:05.608452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea73c8-0ca3-42ca-8d6b-94084cfc33ee admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.611197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 793/1588] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:05.612099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35636a-066d-44aa-8ffc-4c3644145298 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.621847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea73c8-0ca3-42ca-8d6b-94084cfc33ee admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.623444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35636a-066d-44aa-8ffc-4c3644145298 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.627674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603abb39-cc93-4c14-b1cc-e63220a50407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.628605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603abb39-cc93-4c14-b1cc-e63220a50407 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.628828 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f442c93-2e3d-4c6f-beec-7662b22e8ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.629261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603abb39-cc93-4c14-b1cc-e63220a50407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.629706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f442c93-2e3d-4c6f-beec-7662b22e8ded admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.629957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603abb39-cc93-4c14-b1cc-e63220a50407 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.630116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f442c93-2e3d-4c6f-beec-7662b22e8ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.630914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f442c93-2e3d-4c6f-beec-7662b22e8ded admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.631326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603abb39-cc93-4c14-b1cc-e63220a50407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.632198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f442c93-2e3d-4c6f-beec-7662b22e8ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.634104 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f442c93-2e3d-4c6f-beec-7662b22e8ded admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.634762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603abb39-cc93-4c14-b1cc-e63220a50407 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.647758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 794/1589] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:05.648792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f442c93-2e3d-4c6f-beec-7662b22e8ded admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.663679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 796/1590] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:05 2022] POST /identity/v3/users => generated 303 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:05.665229 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603abb39-cc93-4c14-b1cc-e63220a50407 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.668100 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f442c93-2e3d-4c6f-beec-7662b22e8ded admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.673153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:05.675094 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e90165-a228-4595-a4b0-8e37d5ba98f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.675847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e90165-a228-4595-a4b0-8e37d5ba98f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.676251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e90165-a228-4595-a4b0-8e37d5ba98f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.676604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e90165-a228-4595-a4b0-8e37d5ba98f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.677451 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e90165-a228-4595-a4b0-8e37d5ba98f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.679361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e90165-a228-4595-a4b0-8e37d5ba98f6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.679781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603abb39-cc93-4c14-b1cc-e63220a50407 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.688646 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68423514-97f8-4601-8634-bcf6174c398a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.689552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68423514-97f8-4601-8634-bcf6174c398a admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.690062 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68423514-97f8-4601-8634-bcf6174c398a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.690387 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68423514-97f8-4601-8634-bcf6174c398a admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.691260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 795/1591] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:05.695159 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68423514-97f8-4601-8634-bcf6174c398a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.695324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e90165-a228-4595-a4b0-8e37d5ba98f6 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.695472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68423514-97f8-4601-8634-bcf6174c398a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.708619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 797/1592] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:05.710744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e90165-a228-4595-a4b0-8e37d5ba98f6 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.713477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68423514-97f8-4601-8634-bcf6174c398a admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.718312 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add6c40-c9b8-4430-b17d-9fd711684ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.719626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add6c40-c9b8-4430-b17d-9fd711684ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.720268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add6c40-c9b8-4430-b17d-9fd711684ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.720996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add6c40-c9b8-4430-b17d-9fd711684ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.723594 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add6c40-c9b8-4430-b17d-9fd711684ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.726109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68423514-97f8-4601-8634-bcf6174c398a admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.728444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add6c40-c9b8-4430-b17d-9fd711684ba6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.732624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1af32-0b69-4de4-9259-c5d631f020a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.733535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1af32-0b69-4de4-9259-c5d631f020a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.734033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1af32-0b69-4de4-9259-c5d631f020a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.734429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1af32-0b69-4de4-9259-c5d631f020a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.735271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1af32-0b69-4de4-9259-c5d631f020a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.737905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1af32-0b69-4de4-9259-c5d631f020a4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.743797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 796/1593] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:05.745309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add6c40-c9b8-4430-b17d-9fd711684ba6 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.751762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 798/1594] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:05.752807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1af32-0b69-4de4-9259-c5d631f020a4 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.762302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add6c40-c9b8-4430-b17d-9fd711684ba6 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.768173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1af32-0b69-4de4-9259-c5d631f020a4 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.768447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aec782-c4d1-47ad-bd4a-59083d8f2f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.769795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aec782-c4d1-47ad-bd4a-59083d8f2f3b admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.770091 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aec782-c4d1-47ad-bd4a-59083d8f2f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.770593 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aec782-c4d1-47ad-bd4a-59083d8f2f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.771528 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aec782-c4d1-47ad-bd4a-59083d8f2f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.774400 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aec782-c4d1-47ad-bd4a-59083d8f2f3b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.774563 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e1913-e5d2-4965-a841-c8f5c2a767f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.775316 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e1913-e5d2-4965-a841-c8f5c2a767f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.775702 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e1913-e5d2-4965-a841-c8f5c2a767f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.776272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e1913-e5d2-4965-a841-c8f5c2a767f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.777524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e1913-e5d2-4965-a841-c8f5c2a767f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.779401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e1913-e5d2-4965-a841-c8f5c2a767f6 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.791375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 797/1595] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:05.792364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aec782-c4d1-47ad-bd4a-59083d8f2f3b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.793286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 799/1596] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:05.794158 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e1913-e5d2-4965-a841-c8f5c2a767f6 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.807235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aec782-c4d1-47ad-bd4a-59083d8f2f3b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.809196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e1913-e5d2-4965-a841-c8f5c2a767f6 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.815704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719bf6c-3eed-4574-b908-c109600cc33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.816696 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719bf6c-3eed-4574-b908-c109600cc33d admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.816814 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d5a54-70e8-4f64-b195-49d45ecdce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.817292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719bf6c-3eed-4574-b908-c109600cc33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.817751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719bf6c-3eed-4574-b908-c109600cc33d admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.817914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d5a54-70e8-4f64-b195-49d45ecdce64 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.818204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d5a54-70e8-4f64-b195-49d45ecdce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.818478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d5a54-70e8-4f64-b195-49d45ecdce64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.819188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719bf6c-3eed-4574-b908-c109600cc33d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.819371 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d5a54-70e8-4f64-b195-49d45ecdce64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.821143 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719bf6c-3eed-4574-b908-c109600cc33d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.821838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d5a54-70e8-4f64-b195-49d45ecdce64 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.835617 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 798/1598] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:05.835820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 800/1598] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:05.836829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719bf6c-3eed-4574-b908-c109600cc33d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.837280 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d5a54-70e8-4f64-b195-49d45ecdce64 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.849563 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d5a54-70e8-4f64-b195-49d45ecdce64 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.856470 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e4532-db7c-4b5b-9243-36114a0e5d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.857055 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719bf6c-3eed-4574-b908-c109600cc33d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.857395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e4532-db7c-4b5b-9243-36114a0e5d34 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.857701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e4532-db7c-4b5b-9243-36114a0e5d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.857958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e4532-db7c-4b5b-9243-36114a0e5d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.859140 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e4532-db7c-4b5b-9243-36114a0e5d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.860987 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e4532-db7c-4b5b-9243-36114a0e5d34 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.863883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5f2b9-b4a4-436f-87bc-0bb09e6289b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.864684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5f2b9-b4a4-436f-87bc-0bb09e6289b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.864976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5f2b9-b4a4-436f-87bc-0bb09e6289b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.865385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5f2b9-b4a4-436f-87bc-0bb09e6289b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.866604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe5f2b9-b4a4-436f-87bc-0bb09e6289b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.869830 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe5f2b9-b4a4-436f-87bc-0bb09e6289b3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.875941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 801/1599] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:05.875941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e4532-db7c-4b5b-9243-36114a0e5d34 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.890370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e4532-db7c-4b5b-9243-36114a0e5d34 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.904352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 799/1600] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:05.906690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5f2b9-b4a4-436f-87bc-0bb09e6289b3 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.906937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b388845-9a06-4503-9573-b0521b49a29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.907921 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388845-9a06-4503-9573-b0521b49a29e admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.908209 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388845-9a06-4503-9573-b0521b49a29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.908483 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388845-9a06-4503-9573-b0521b49a29e admin admin] PATH_INFO: `/v3/projects/956d6e9aeb1c4279955c146130b1fb7b/users/83cd7641aaa546edb07003ba353e125c/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.919891 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5f2b9-b4a4-436f-87bc-0bb09e6289b3 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:05.926543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe87c49-32f8-4073-a283-66d8b47f5871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:05.927233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe87c49-32f8-4073-a283-66d8b47f5871 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:05.927521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe87c49-32f8-4073-a283-66d8b47f5871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:05.928261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe87c49-32f8-4073-a283-66d8b47f5871 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:05.929702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe87c49-32f8-4073-a283-66d8b47f5871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.930630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b388845-9a06-4503-9573-b0521b49a29e admin admin] RBAC: Authorizing `identity:create_grant(project_id=956d6e9aeb1c4279955c146130b1fb7b, user_id=83cd7641aaa546edb07003ba353e125c, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:05.931559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe87c49-32f8-4073-a283-66d8b47f5871 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.933726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b388845-9a06-4503-9573-b0521b49a29e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:05.943048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 800/1601] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:05.947011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe87c49-32f8-4073-a283-66d8b47f5871 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.954431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 802/1602] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:05 2022] PUT /identity/v3/projects/956d6e9aeb1c4279955c146130b1fb7b/users/83cd7641aaa546edb07003ba353e125c/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:05.955389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b388845-9a06-4503-9573-b0521b49a29e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:05.968365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe87c49-32f8-4073-a283-66d8b47f5871 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:06.045820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b388845-9a06-4503-9573-b0521b49a29e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:06.061966 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5297eb-c0d1-48ac-bb7e-c8b91924d4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:06.064272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5297eb-c0d1-48ac-bb7e-c8b91924d4b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.068059 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5297eb-c0d1-48ac-bb7e-c8b91924d4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.068420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5297eb-c0d1-48ac-bb7e-c8b91924d4b4 admin admin] PATH_INFO: `/v3/projects/ba94c7ecfaf447c79c89ad09c446c522/users/e1214914fade457d9027bb097911920c/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.103508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5297eb-c0d1-48ac-bb7e-c8b91924d4b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba94c7ecfaf447c79c89ad09c446c522, user_id=e1214914fade457d9027bb097911920c, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:06.107519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5297eb-c0d1-48ac-bb7e-c8b91924d4b4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:06.115489 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc0c7c-8fab-48fe-8c47-9e9d58bbb91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:06.119036 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc0c7c-8fab-48fe-8c47-9e9d58bbb91a admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.119874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc0c7c-8fab-48fe-8c47-9e9d58bbb91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.122273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc0c7c-8fab-48fe-8c47-9e9d58bbb91a admin admin] PATH_INFO: `/v3/projects/e0ac172df04f4957bdd5003bb71ce010/users/aa3150d932224bb0a737a3977fff46bc/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.134176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 803/1603] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:05 2022] PUT /identity/v3/projects/ba94c7ecfaf447c79c89ad09c446c522/users/e1214914fade457d9027bb097911920c/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:06.135178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5297eb-c0d1-48ac-bb7e-c8b91924d4b4 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:06.169328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc0c7c-8fab-48fe-8c47-9e9d58bbb91a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ac172df04f4957bdd5003bb71ce010, user_id=aa3150d932224bb0a737a3977fff46bc, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:06.176423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc0c7c-8fab-48fe-8c47-9e9d58bbb91a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:06.189621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:06.197511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 801/1604] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:05 2022] PUT /identity/v3/projects/e0ac172df04f4957bdd5003bb71ce010/users/aa3150d932224bb0a737a3977fff46bc/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:06.202996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc0c7c-8fab-48fe-8c47-9e9d58bbb91a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:06.285967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5297eb-c0d1-48ac-bb7e-c8b91924d4b4 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:06.301947 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:06.303439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc0c7c-8fab-48fe-8c47-9e9d58bbb91a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:06.311766 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d42432-3e4c-43c2-9df3-de7644b0c906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:06.313925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d42432-3e4c-43c2-9df3-de7644b0c906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.315420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d42432-3e4c-43c2-9df3-de7644b0c906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.316552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d42432-3e4c-43c2-9df3-de7644b0c906 admin admin] PATH_INFO: `/v3/projects/e10990abd8864745b2766df2f30a4bc8/users/4f603b289d964ee3ba3931948acf6788/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.339675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d42432-3e4c-43c2-9df3-de7644b0c906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e10990abd8864745b2766df2f30a4bc8, user_id=4f603b289d964ee3ba3931948acf6788, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:06.342906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d42432-3e4c-43c2-9df3-de7644b0c906 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:06.361943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dfeb5b-f9cb-4429-b281-ae2197e3a030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:06.362671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfeb5b-f9cb-4429-b281-ae2197e3a030 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.362963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfeb5b-f9cb-4429-b281-ae2197e3a030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.363240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfeb5b-f9cb-4429-b281-ae2197e3a030 admin admin] PATH_INFO: `/v3/projects/7cdf56452f2f49d792327ead98000779/users/391da92d8f2f43aa97ad11e97c482e35/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.374698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 802/1605] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:06 2022] PUT /identity/v3/projects/e10990abd8864745b2766df2f30a4bc8/users/4f603b289d964ee3ba3931948acf6788/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:06.375903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d42432-3e4c-43c2-9df3-de7644b0c906 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:06.393376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dfeb5b-f9cb-4429-b281-ae2197e3a030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cdf56452f2f49d792327ead98000779, user_id=391da92d8f2f43aa97ad11e97c482e35, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:06.399309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dfeb5b-f9cb-4429-b281-ae2197e3a030 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:06.418435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 804/1606] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:06 2022] PUT /identity/v3/projects/7cdf56452f2f49d792327ead98000779/users/391da92d8f2f43aa97ad11e97c482e35/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:06.420002 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af28dfb-0a35-46b8-a81c-4f2f1c2608fd None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.420429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af28dfb-0a35-46b8-a81c-4f2f1c2608fd None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.421372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af28dfb-0a35-46b8-a81c-4f2f1c2608fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.453180 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-4af28dfb-0a35-46b8-a81c-4f2f1c2608fd None None] MFA Rules not processed for user `83cd7641aaa546edb07003ba353e125c`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:06.455226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4af28dfb-0a35-46b8-a81c-4f2f1c2608fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:06.479368 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d42432-3e4c-43c2-9df3-de7644b0c906 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:06.536884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 805/1607] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:06.554786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c3bf1-4d90-44a9-bf84-85b21d0f6cd3 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.555072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c3bf1-4d90-44a9-bf84-85b21d0f6cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.557716 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c3bf1-4d90-44a9-bf84-85b21d0f6cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.592960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fcb5f8-3f14-4130-98bf-1d849f782da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:06.594712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcb5f8-3f14-4130-98bf-1d849f782da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.595385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcb5f8-3f14-4130-98bf-1d849f782da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.595534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcb5f8-3f14-4130-98bf-1d849f782da1 admin admin] PATH_INFO: `/v3/projects/c553a9f4679e4d6f9f7e23cb5d1f652f/users/f90fa3b5a1d5480aa1b81f06cc4a6ed6/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.614814 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-c62c3bf1-4d90-44a9-bf84-85b21d0f6cd3 None None] MFA Rules not processed for user `e1214914fade457d9027bb097911920c`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:06.616930 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c62c3bf1-4d90-44a9-bf84-85b21d0f6cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:06.624031 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fcb5f8-3f14-4130-98bf-1d849f782da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c553a9f4679e4d6f9f7e23cb5d1f652f, user_id=f90fa3b5a1d5480aa1b81f06cc4a6ed6, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:06.627818 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fcb5f8-3f14-4130-98bf-1d849f782da1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:06.650227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 803/1608] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:06 2022] PUT /identity/v3/projects/c553a9f4679e4d6f9f7e23cb5d1f652f/users/f90fa3b5a1d5480aa1b81f06cc4a6ed6/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:06.651961 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722eecb-098a-454b-b38d-005ee093ffc8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.652336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722eecb-098a-454b-b38d-005ee093ffc8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.652610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722eecb-098a-454b-b38d-005ee093ffc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.682335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e722eecb-098a-454b-b38d-005ee093ffc8 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:06.684175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e722eecb-098a-454b-b38d-005ee093ffc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:06.697213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:06.724511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 806/1609] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:06 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:06.727292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e0d38-d804-4325-a9fe-2f9872462713 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.727689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e0d38-d804-4325-a9fe-2f9872462713 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.728079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e0d38-d804-4325-a9fe-2f9872462713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.757868 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-5f0e0d38-d804-4325-a9fe-2f9872462713 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:06.759878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5f0e0d38-d804-4325-a9fe-2f9872462713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:06.777053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:06.779716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 804/1610] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:06.784470 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe3ec7-8f50-4faf-967e-9ce05f101e09 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.784906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe3ec7-8f50-4faf-967e-9ce05f101e09 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.785241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe3ec7-8f50-4faf-967e-9ce05f101e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.837981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-fcbe3ec7-8f50-4faf-967e-9ce05f101e09 None None] MFA Rules not processed for user `391da92d8f2f43aa97ad11e97c482e35`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:06.840091 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fcbe3ec7-8f50-4faf-967e-9ce05f101e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:06.864992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 807/1611] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:06.864992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b3868-0ef2-442e-8f11-613f968b713c None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.864992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b3868-0ef2-442e-8f11-613f968b713c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.865438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b3868-0ef2-442e-8f11-613f968b713c None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.868136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 808/1612] 10.222.0.39 () {54 vars in 961 bytes} [Tue Jun 7 08:36:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:06.869405 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da27b1-24c3-4607-932f-fad3b99209fb None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.869849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da27b1-24c3-4607-932f-fad3b99209fb None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.870299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da27b1-24c3-4607-932f-fad3b99209fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.902874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-17da27b1-24c3-4607-932f-fad3b99209fb None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:06.904316 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-17da27b1-24c3-4607-932f-fad3b99209fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:06.921688 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 809/1613] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:06.922880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da27b1-24c3-4607-932f-fad3b99209fb None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:06.938929 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da27b1-24c3-4607-932f-fad3b99209fb None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:06.948168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f295aa-6595-4e32-a08b-0de7055955c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:06.949011 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f295aa-6595-4e32-a08b-0de7055955c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.949386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f295aa-6595-4e32-a08b-0de7055955c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.950309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f295aa-6595-4e32-a08b-0de7055955c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.951126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f295aa-6595-4e32-a08b-0de7055955c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:06.955342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f295aa-6595-4e32-a08b-0de7055955c8 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:06.976153 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 805/1614] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:06 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:06.977019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe3ec7-8f50-4faf-967e-9ce05f101e09 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:07.001747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 810/1615] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:06 2022] POST /identity/v3/projects => generated 367 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:07.003255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4db76c-1e4d-4a6d-8ecb-8ce5c6eb87f6 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.003553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4db76c-1e4d-4a6d-8ecb-8ce5c6eb87f6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.003836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4db76c-1e4d-4a6d-8ecb-8ce5c6eb87f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.060424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-ba4db76c-1e4d-4a6d-8ecb-8ce5c6eb87f6 None None] MFA Rules not processed for user `e61d179002a447d89160d8cb435089eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:07.063577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ba4db76c-1e4d-4a6d-8ecb-8ce5c6eb87f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:07.081401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe3ec7-8f50-4faf-967e-9ce05f101e09 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:07.163747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 811/1616] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:07.164729 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4db76c-1e4d-4a6d-8ecb-8ce5c6eb87f6 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:07.166093 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bed85-571e-4cf4-be86-0b8eca8a4a2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:07.167413 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bed85-571e-4cf4-be86-0b8eca8a4a2f service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.167810 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bed85-571e-4cf4-be86-0b8eca8a4a2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.168195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bed85-571e-4cf4-be86-0b8eca8a4a2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.188114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4db76c-1e4d-4a6d-8ecb-8ce5c6eb87f6 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:07.193601 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:07.197388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53c77f-ea38-4ac2-a60e-f1024b49a48a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:07.198126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53c77f-ea38-4ac2-a60e-f1024b49a48a service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.198466 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53c77f-ea38-4ac2-a60e-f1024b49a48a service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.198751 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53c77f-ea38-4ac2-a60e-f1024b49a48a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.275013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bed85-571e-4cf4-be86-0b8eca8a4a2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:07.278334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bed85-571e-4cf4-be86-0b8eca8a4a2f service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:07.299198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53c77f-ea38-4ac2-a60e-f1024b49a48a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:07.301190 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53c77f-ea38-4ac2-a60e-f1024b49a48a service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:07.308040 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 806/1617] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:07.312152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ed47d-2b01-498c-9ed0-ea0a01da858d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.312499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ed47d-2b01-498c-9ed0-ea0a01da858d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.313326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ed47d-2b01-498c-9ed0-ea0a01da858d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.316888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 812/1618] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:07.318249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53c77f-ea38-4ac2-a60e-f1024b49a48a service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:07.329006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53c77f-ea38-4ac2-a60e-f1024b49a48a service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:07.335058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b87a8e-b559-49f1-93c5-e6d2010a350e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:07.336551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b87a8e-b559-49f1-93c5-e6d2010a350e service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.336551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b87a8e-b559-49f1-93c5-e6d2010a350e service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.336551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b87a8e-b559-49f1-93c5-e6d2010a350e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.348303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5c2ed47d-2b01-498c-9ed0-ea0a01da858d None None] MFA Rules not processed for user `3f2a2bf913d1412f996acf4401ba7673`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:07.349917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5c2ed47d-2b01-498c-9ed0-ea0a01da858d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:07.367865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b87a8e-b559-49f1-93c5-e6d2010a350e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:07.370101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b87a8e-b559-49f1-93c5-e6d2010a350e service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:07.377982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:07.388880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 813/1619] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:07.389782 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b87a8e-b559-49f1-93c5-e6d2010a350e service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:07.408845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b87a8e-b559-49f1-93c5-e6d2010a350e service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:07.422559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e586481-7f43-49be-82af-b17751847bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:07.423479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e586481-7f43-49be-82af-b17751847bfc admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.423767 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e586481-7f43-49be-82af-b17751847bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.424196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e586481-7f43-49be-82af-b17751847bfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.425591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e586481-7f43-49be-82af-b17751847bfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:07.427602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e586481-7f43-49be-82af-b17751847bfc admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:07.495309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 814/1620] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:07 2022] POST /identity/v3/projects => generated 403 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:07.495309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e586481-7f43-49be-82af-b17751847bfc admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:07.495598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 807/1621] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Jun 7 08:36:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:07.498096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebc071-d95a-4d25-9cb0-ec1819ab4f31 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.498557 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebc071-d95a-4d25-9cb0-ec1819ab4f31 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.498941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebc071-d95a-4d25-9cb0-ec1819ab4f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.530901 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e7ebc071-d95a-4d25-9cb0-ec1819ab4f31 None None] MFA Rules not processed for user `e61d179002a447d89160d8cb435089eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:07.532889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e7ebc071-d95a-4d25-9cb0-ec1819ab4f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:07.611995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e586481-7f43-49be-82af-b17751847bfc admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:07.622331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253612b7-6c9c-411c-b447-c6437038e7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:07.623607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253612b7-6c9c-411c-b447-c6437038e7ad admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.624195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253612b7-6c9c-411c-b447-c6437038e7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.624784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253612b7-6c9c-411c-b447-c6437038e7ad admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.626014 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253612b7-6c9c-411c-b447-c6437038e7ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:07.629286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253612b7-6c9c-411c-b447-c6437038e7ad admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:07.648868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 808/1622] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:07.650523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ebc071-d95a-4d25-9cb0-ec1819ab4f31 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:07.669575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 815/1623] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:07 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:07.669575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253612b7-6c9c-411c-b447-c6437038e7ad admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:07.703507 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253612b7-6c9c-411c-b447-c6437038e7ad admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:07.719926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084155a-1b3b-4a76-80dd-7f6a6ef1e7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:07.722998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084155a-1b3b-4a76-80dd-7f6a6ef1e7e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.723398 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084155a-1b3b-4a76-80dd-7f6a6ef1e7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.723807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084155a-1b3b-4a76-80dd-7f6a6ef1e7e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.724535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084155a-1b3b-4a76-80dd-7f6a6ef1e7e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:07.739402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084155a-1b3b-4a76-80dd-7f6a6ef1e7e9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:07.778597 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:07.796644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 816/1624] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:07 2022] POST /identity/v3/users => generated 308 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:07.797567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084155a-1b3b-4a76-80dd-7f6a6ef1e7e9 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:07.817464 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084155a-1b3b-4a76-80dd-7f6a6ef1e7e9 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:07.829598 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e869644-71f4-4810-b265-63c703725fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:07.830926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e869644-71f4-4810-b265-63c703725fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.831231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e869644-71f4-4810-b265-63c703725fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.831516 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e869644-71f4-4810-b265-63c703725fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.832267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e869644-71f4-4810-b265-63c703725fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:07.834182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e869644-71f4-4810-b265-63c703725fa1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:07.834603 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ebc071-d95a-4d25-9cb0-ec1819ab4f31 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:07.843413 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169609c-8a06-4238-b366-14a56ceddbdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:07.844656 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169609c-8a06-4238-b366-14a56ceddbdc service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.845671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169609c-8a06-4238-b366-14a56ceddbdc service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.846145 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169609c-8a06-4238-b366-14a56ceddbdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.855053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 817/1625] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:07.855053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919a37e-940e-45c4-af43-914b10514479 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.855053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919a37e-940e-45c4-af43-914b10514479 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.855053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919a37e-940e-45c4-af43-914b10514479 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.858763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 818/1626] 10.222.0.39 () {54 vars in 961 bytes} [Tue Jun 7 08:36:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:07.859488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919a37e-940e-45c4-af43-914b10514479 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:07.874703 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919a37e-940e-45c4-af43-914b10514479 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:07.880243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a27b1a-7428-48f4-b106-78034783020a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:07.880926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a27b1a-7428-48f4-b106-78034783020a admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.881301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a27b1a-7428-48f4-b106-78034783020a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.882359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a27b1a-7428-48f4-b106-78034783020a admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.883307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a27b1a-7428-48f4-b106-78034783020a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:07.888039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a27b1a-7428-48f4-b106-78034783020a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:07.901237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 819/1627] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:07.905978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a27b1a-7428-48f4-b106-78034783020a admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:07.918768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a27b1a-7428-48f4-b106-78034783020a admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:07.924215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91590af9-1773-4282-99ff-e694a8ce7830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:07.925492 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91590af9-1773-4282-99ff-e694a8ce7830 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:07.925778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91590af9-1773-4282-99ff-e694a8ce7830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:07.926057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91590af9-1773-4282-99ff-e694a8ce7830 admin admin] PATH_INFO: `/v3/projects/ec2b708c4e2d45aaaa213af9c05c359f/users/ba17e70e69b2460bb0bdb9383e482ec6/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:07.954893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91590af9-1773-4282-99ff-e694a8ce7830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2b708c4e2d45aaaa213af9c05c359f, user_id=ba17e70e69b2460bb0bdb9383e482ec6, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:07.959418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91590af9-1773-4282-99ff-e694a8ce7830 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:07.979503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169609c-8a06-4238-b366-14a56ceddbdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:07.985817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169609c-8a06-4238-b366-14a56ceddbdc service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:08.007550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 820/1628] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:07 2022] PUT /identity/v3/projects/ec2b708c4e2d45aaaa213af9c05c359f/users/ba17e70e69b2460bb0bdb9383e482ec6/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:08.007550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b46c8d-4e93-42e8-a17d-d12eead8e04b None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:08.007550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b46c8d-4e93-42e8-a17d-d12eead8e04b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:08.007550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b46c8d-4e93-42e8-a17d-d12eead8e04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:08.017193 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 809/1629] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:08.021407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169609c-8a06-4238-b366-14a56ceddbdc service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:08.040452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e3b46c8d-4e93-42e8-a17d-d12eead8e04b None None] MFA Rules not processed for user `3f2a2bf913d1412f996acf4401ba7673`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:08.042537 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e3b46c8d-4e93-42e8-a17d-d12eead8e04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:08.044894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169609c-8a06-4238-b366-14a56ceddbdc service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:08.172201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:08.207076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98a2ca-be93-4022-844a-46b59b6af756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:08.213637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98a2ca-be93-4022-844a-46b59b6af756 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:08.214980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98a2ca-be93-4022-844a-46b59b6af756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:08.217384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98a2ca-be93-4022-844a-46b59b6af756 admin admin] PATH_INFO: `/v3/projects/54b04e59a81a4a659dafabbebaf44b50/users/d195c70b52ec47e18ba224d90687905b/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:08.245197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 821/1630] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Jun 7 08:36:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:08.246827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b46c8d-4e93-42e8-a17d-d12eead8e04b None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:08.272806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98a2ca-be93-4022-844a-46b59b6af756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54b04e59a81a4a659dafabbebaf44b50, user_id=d195c70b52ec47e18ba224d90687905b, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:08.279989 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98a2ca-be93-4022-844a-46b59b6af756 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:08.280259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b46c8d-4e93-42e8-a17d-d12eead8e04b None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:08.294344 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa3853-7466-4bfe-8677-ff8196a7c02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:08.295215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa3853-7466-4bfe-8677-ff8196a7c02e admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:08.295975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa3853-7466-4bfe-8677-ff8196a7c02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:08.296559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa3853-7466-4bfe-8677-ff8196a7c02e admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:08.297602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa3853-7466-4bfe-8677-ff8196a7c02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:08.308964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa3853-7466-4bfe-8677-ff8196a7c02e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:08.320831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 810/1631] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:08 2022] PUT /identity/v3/projects/54b04e59a81a4a659dafabbebaf44b50/users/d195c70b52ec47e18ba224d90687905b/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:08.324679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98a2ca-be93-4022-844a-46b59b6af756 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:08.349634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 822/1632] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:08.349634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa3853-7466-4bfe-8677-ff8196a7c02e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:08.383153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa3853-7466-4bfe-8677-ff8196a7c02e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:08.522643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:08.539526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98a2ca-be93-4022-844a-46b59b6af756 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:08.557847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e5365-a17e-412b-833b-021a946f4f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:08.558255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301bc61-2ddb-4536-ab5a-97c603f2b6ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:08.559231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301bc61-2ddb-4536-ab5a-97c603f2b6ea service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:08.562125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e5365-a17e-412b-833b-021a946f4f16 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:08.562435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e5365-a17e-412b-833b-021a946f4f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:08.562702 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e5365-a17e-412b-833b-021a946f4f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:08.563481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e5365-a17e-412b-833b-021a946f4f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:08.564630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301bc61-2ddb-4536-ab5a-97c603f2b6ea service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:08.565008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301bc61-2ddb-4536-ab5a-97c603f2b6ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:08.568438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e5365-a17e-412b-833b-021a946f4f16 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:08.604419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 823/1633] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:08.605231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e5365-a17e-412b-833b-021a946f4f16 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:08.636550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e5365-a17e-412b-833b-021a946f4f16 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:08.648334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775acff9-461a-43b6-8850-884f3d5e91ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:08.650124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775acff9-461a-43b6-8850-884f3d5e91ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:08.650605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775acff9-461a-43b6-8850-884f3d5e91ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:08.657739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775acff9-461a-43b6-8850-884f3d5e91ef admin admin] PATH_INFO: `/v3/projects/ec2b708c4e2d45aaaa213af9c05c359f/users/ba17e70e69b2460bb0bdb9383e482ec6/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:08.663666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775acff9-461a-43b6-8850-884f3d5e91ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2b708c4e2d45aaaa213af9c05c359f, user_id=ba17e70e69b2460bb0bdb9383e482ec6, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:08.668070 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775acff9-461a-43b6-8850-884f3d5e91ef admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:08.691546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301bc61-2ddb-4536-ab5a-97c603f2b6ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:08.693272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301bc61-2ddb-4536-ab5a-97c603f2b6ea service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:08.701936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 824/1634] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:08 2022] PUT /identity/v3/projects/ec2b708c4e2d45aaaa213af9c05c359f/users/ba17e70e69b2460bb0bdb9383e482ec6/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:08.703202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775acff9-461a-43b6-8850-884f3d5e91ef admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:08.710425 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 811/1635] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:08.713960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423bc4-cc02-4b88-99ff-2dd566dc51eb None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:08.714846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423bc4-cc02-4b88-99ff-2dd566dc51eb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:08.715522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423bc4-cc02-4b88-99ff-2dd566dc51eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:08.747869 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0c423bc4-cc02-4b88-99ff-2dd566dc51eb None None] MFA Rules not processed for user `ba17e70e69b2460bb0bdb9383e482ec6`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:08.749506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0c423bc4-cc02-4b88-99ff-2dd566dc51eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:08.795505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775acff9-461a-43b6-8850-884f3d5e91ef admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:08.801786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e357fe-db4d-47cb-bdb8-925693ee6921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:08.803460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e357fe-db4d-47cb-bdb8-925693ee6921 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:08.803995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e357fe-db4d-47cb-bdb8-925693ee6921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:08.804518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e357fe-db4d-47cb-bdb8-925693ee6921 admin admin] PATH_INFO: `/v3/projects/54b04e59a81a4a659dafabbebaf44b50/users/d195c70b52ec47e18ba224d90687905b/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:08.810761 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e357fe-db4d-47cb-bdb8-925693ee6921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54b04e59a81a4a659dafabbebaf44b50, user_id=d195c70b52ec47e18ba224d90687905b, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:08.814590 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e357fe-db4d-47cb-bdb8-925693ee6921 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:08.831219 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 812/1636] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:08 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:08.833318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89213f44-3ea0-4a53-8667-cba3b39c108e None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:08.833623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89213f44-3ea0-4a53-8667-cba3b39c108e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:08.833903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89213f44-3ea0-4a53-8667-cba3b39c108e None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:08.835776 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 813/1637] 10.222.0.39 () {54 vars in 912 bytes} [Tue Jun 7 08:36:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:08.841425 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e951e82-0265-4abd-b41c-01021243dcb3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:08.841720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e951e82-0265-4abd-b41c-01021243dcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:08.842006 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e951e82-0265-4abd-b41c-01021243dcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:08.854277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 825/1638] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:08 2022] PUT /identity/v3/projects/54b04e59a81a4a659dafabbebaf44b50/users/d195c70b52ec47e18ba224d90687905b/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:08.859188 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f4b21-27a3-48fd-9980-10423ee036f1 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:08.859717 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f4b21-27a3-48fd-9980-10423ee036f1 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:08.860066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f4b21-27a3-48fd-9980-10423ee036f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:08.878903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3e951e82-0265-4abd-b41c-01021243dcb3 None None] MFA Rules not processed for user `3f2a2bf913d1412f996acf4401ba7673`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:08.882449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3e951e82-0265-4abd-b41c-01021243dcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:08.932830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-fc2f4b21-27a3-48fd-9980-10423ee036f1 None None] MFA Rules not processed for user `d195c70b52ec47e18ba224d90687905b`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:08.936436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-fc2f4b21-27a3-48fd-9980-10423ee036f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:09.031380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:09.070863 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 826/1639] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:08 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:09.071841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f4b21-27a3-48fd-9980-10423ee036f1 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:09.081605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 814/1640] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jun 7 08:36:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:09.085970 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e951e82-0265-4abd-b41c-01021243dcb3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:09.129588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:09.179694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e951e82-0265-4abd-b41c-01021243dcb3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:09.188064 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a8e6e-e881-44f6-9cbf-8170c690e76b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:09.188455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a8e6e-e881-44f6-9cbf-8170c690e76b service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:09.191721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a8e6e-e881-44f6-9cbf-8170c690e76b service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:09.192129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a8e6e-e881-44f6-9cbf-8170c690e76b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:09.192778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f4b21-27a3-48fd-9980-10423ee036f1 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:09.203502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ee31e-710d-416d-965a-1dba870a2b55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:09.204615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ee31e-710d-416d-965a-1dba870a2b55 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:09.205341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ee31e-710d-416d-965a-1dba870a2b55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:09.205862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ee31e-710d-416d-965a-1dba870a2b55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:09.213758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a8e6e-e881-44f6-9cbf-8170c690e76b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:09.215725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a8e6e-e881-44f6-9cbf-8170c690e76b service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:09.231035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 815/1641] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:09 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:09.339069 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a8e6e-e881-44f6-9cbf-8170c690e76b service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:09.353558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a8e6e-e881-44f6-9cbf-8170c690e76b service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:09.370362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42848ceb-03db-4a67-8adf-167eaef92fec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:09.371691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42848ceb-03db-4a67-8adf-167eaef92fec service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:09.371984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42848ceb-03db-4a67-8adf-167eaef92fec service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:09.372853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42848ceb-03db-4a67-8adf-167eaef92fec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:09.386162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ee31e-710d-416d-965a-1dba870a2b55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:09.388262 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ee31e-710d-416d-965a-1dba870a2b55 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:09.391828 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42848ceb-03db-4a67-8adf-167eaef92fec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:09.393382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42848ceb-03db-4a67-8adf-167eaef92fec service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:09.409197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 827/1642] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:09.409368 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 816/1643] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:09.417889 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aab16-f488-4abf-9cc4-96c1b5759dc9 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:09.418270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aab16-f488-4abf-9cc4-96c1b5759dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:09.418663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aab16-f488-4abf-9cc4-96c1b5759dc9 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:09.420568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 828/1644] 10.222.0.39 () {54 vars in 912 bytes} [Tue Jun 7 08:36:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:09.426104 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743e8da-eff8-47ee-9c9e-74dcc7360b6b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:09.427798 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743e8da-eff8-47ee-9c9e-74dcc7360b6b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:09.428543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743e8da-eff8-47ee-9c9e-74dcc7360b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:09.504480 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b743e8da-eff8-47ee-9c9e-74dcc7360b6b None None] MFA Rules not processed for user `3f2a2bf913d1412f996acf4401ba7673`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:09.507993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b743e8da-eff8-47ee-9c9e-74dcc7360b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:09.533591 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 817/1645] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jun 7 08:36:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:09.545752 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382aab16-f488-4abf-9cc4-96c1b5759dc9 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:09.547472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-382aab16-f488-4abf-9cc4-96c1b5759dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:09.579681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743e8da-eff8-47ee-9c9e-74dcc7360b6b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:09.614966 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743e8da-eff8-47ee-9c9e-74dcc7360b6b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:09.632036 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169dbb5f-734b-48a2-968e-08df25fda21d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:09.633321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dbb5f-734b-48a2-968e-08df25fda21d service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:09.633905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dbb5f-734b-48a2-968e-08df25fda21d service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:09.634284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dbb5f-734b-48a2-968e-08df25fda21d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:09.704806 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382aab16-f488-4abf-9cc4-96c1b5759dc9 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:09.722058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca9ab1-f0cc-4c7a-8e59-b6796d1bede3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e4531ac696445ba5bfa9870984befd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:09.722742 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca9ab1-f0cc-4c7a-8e59-b6796d1bede3 service glance] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:09.723030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca9ab1-f0cc-4c7a-8e59-b6796d1bede3 service glance] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:09.723309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca9ab1-f0cc-4c7a-8e59-b6796d1bede3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:09.759032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca9ab1-f0cc-4c7a-8e59-b6796d1bede3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:09.760842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca9ab1-f0cc-4c7a-8e59-b6796d1bede3 service glance] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:09.783135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 829/1646] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:09.793787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169dbb5f-734b-48a2-968e-08df25fda21d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:09.795761 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169dbb5f-734b-48a2-968e-08df25fda21d service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:09.827253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 818/1647] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:10.018284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca9ab1-f0cc-4c7a-8e59-b6796d1bede3 service glance] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:10.043969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169dbb5f-734b-48a2-968e-08df25fda21d service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:10.116083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:10.172262 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca9ab1-f0cc-4c7a-8e59-b6796d1bede3 service glance] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:10.189852 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e4341-2b24-4570-9bb7-2eda3558f095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:10.192198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169dbb5f-734b-48a2-968e-08df25fda21d service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:10.193305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e4341-2b24-4570-9bb7-2eda3558f095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:10.193760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e4341-2b24-4570-9bb7-2eda3558f095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:10.194956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e4341-2b24-4570-9bb7-2eda3558f095 admin admin] PATH_INFO: `/v3/users/83cd7641aaa546edb07003ba353e125c` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:10.196729 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e4341-2b24-4570-9bb7-2eda3558f095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83cd7641aaa546edb07003ba353e125c)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:10.198705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e4341-2b24-4570-9bb7-2eda3558f095 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:10.274165 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143330c-5b38-4bbc-8a3f-732895467e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:10.274165 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143330c-5b38-4bbc-8a3f-732895467e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:10.274165 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143330c-5b38-4bbc-8a3f-732895467e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:10.274165 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143330c-5b38-4bbc-8a3f-732895467e36 admin admin] PATH_INFO: `/v3/users/391da92d8f2f43aa97ad11e97c482e35` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:10.280126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143330c-5b38-4bbc-8a3f-732895467e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=391da92d8f2f43aa97ad11e97c482e35)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:10.280126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: WARNING py.warnings [None req-0143330c-5b38-4bbc-8a3f-732895467e36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:36:10.280126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: warnings.warn(msg) Jun 07 08:36:10.280126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: Jun 07 08:36:10.280126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143330c-5b38-4bbc-8a3f-732895467e36 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:10.333793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4b6e4341-2b24-4570-9bb7-2eda3558f095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83cd7641aaa546edb07003ba353e125c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:10.334908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4b6e4341-2b24-4570-9bb7-2eda3558f095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83cd7641aaa546edb07003ba353e125c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:10.342567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4b6e4341-2b24-4570-9bb7-2eda3558f095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83cd7641aaa546edb07003ba353e125c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:10.360558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4b6e4341-2b24-4570-9bb7-2eda3558f095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83cd7641aaa546edb07003ba353e125c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:10.361323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4b6e4341-2b24-4570-9bb7-2eda3558f095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83cd7641aaa546edb07003ba353e125c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:10.385804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 830/1648] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:10 2022] DELETE /identity/v3/users/83cd7641aaa546edb07003ba353e125c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:10.388173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ed49f-408a-4e0a-bb0c-e2ec626c6e28 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:10.388628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ed49f-408a-4e0a-bb0c-e2ec626c6e28 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:10.389102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ed49f-408a-4e0a-bb0c-e2ec626c6e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:10.408014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:10.412408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-325ed49f-408a-4e0a-bb0c-e2ec626c6e28 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:10.413032 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0143330c-5b38-4bbc-8a3f-732895467e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '391da92d8f2f43aa97ad11e97c482e35'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:10.414073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-325ed49f-408a-4e0a-bb0c-e2ec626c6e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:10.415706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0143330c-5b38-4bbc-8a3f-732895467e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '391da92d8f2f43aa97ad11e97c482e35'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:10.432510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0143330c-5b38-4bbc-8a3f-732895467e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '391da92d8f2f43aa97ad11e97c482e35'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:10.434579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0143330c-5b38-4bbc-8a3f-732895467e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '391da92d8f2f43aa97ad11e97c482e35'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:10.494074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0143330c-5b38-4bbc-8a3f-732895467e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '391da92d8f2f43aa97ad11e97c482e35'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:10.525655 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 831/1649] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:10.527395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdb224-2036-4e17-bc6f-7764d8bcc79f None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:10.527892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdb224-2036-4e17-bc6f-7764d8bcc79f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:10.528380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdb224-2036-4e17-bc6f-7764d8bcc79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:10.560465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 819/1650] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:10 2022] DELETE /identity/v3/users/391da92d8f2f43aa97ad11e97c482e35 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:36:10.580299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335eb4a-b449-4a11-bb50-64bf5ffd40bc None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:10.580596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335eb4a-b449-4a11-bb50-64bf5ffd40bc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:10.580874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335eb4a-b449-4a11-bb50-64bf5ffd40bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:10.585779 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-b7bdb224-2036-4e17-bc6f-7764d8bcc79f None None] MFA Rules not processed for user `f90fa3b5a1d5480aa1b81f06cc4a6ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:10.588512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b7bdb224-2036-4e17-bc6f-7764d8bcc79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:10.657112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:10.689479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 832/1651] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:10 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:10.690442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6335eb4a-b449-4a11-bb50-64bf5ffd40bc None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:10.690568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdb224-2036-4e17-bc6f-7764d8bcc79f None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:10.691680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6335eb4a-b449-4a11-bb50-64bf5ffd40bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:10.692239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b7bdb224-2036-4e17-bc6f-7764d8bcc79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:10.767260 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdb224-2036-4e17-bc6f-7764d8bcc79f None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:10.774457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362addc-66f0-4d60-8557-0a22fb8864db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:10.775493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362addc-66f0-4d60-8557-0a22fb8864db service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:10.776012 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362addc-66f0-4d60-8557-0a22fb8864db service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:10.776377 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362addc-66f0-4d60-8557-0a22fb8864db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:10.837691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 820/1652] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:10.839757 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335eb4a-b449-4a11-bb50-64bf5ffd40bc None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:10.842567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6335eb4a-b449-4a11-bb50-64bf5ffd40bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:10.858222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335eb4a-b449-4a11-bb50-64bf5ffd40bc None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:10.864873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362addc-66f0-4d60-8557-0a22fb8864db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:10.865400 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edbb72-1ef0-440f-b33b-3a0ae81cb2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:10.866993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edbb72-1ef0-440f-b33b-3a0ae81cb2da admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:10.867548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edbb72-1ef0-440f-b33b-3a0ae81cb2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:10.867970 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edbb72-1ef0-440f-b33b-3a0ae81cb2da admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:10.868790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edbb72-1ef0-440f-b33b-3a0ae81cb2da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:10.871676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362addc-66f0-4d60-8557-0a22fb8864db service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:10.872199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edbb72-1ef0-440f-b33b-3a0ae81cb2da admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:10.903452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 833/1653] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:10.931288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362addc-66f0-4d60-8557-0a22fb8864db service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:10.942434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362addc-66f0-4d60-8557-0a22fb8864db service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:10.947171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f752a6-8596-4993-b683-2328752ee600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:10.948333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f752a6-8596-4993-b683-2328752ee600 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:10.948858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f752a6-8596-4993-b683-2328752ee600 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:10.949360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f752a6-8596-4993-b683-2328752ee600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:10.961469 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f752a6-8596-4993-b683-2328752ee600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:10.963191 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f752a6-8596-4993-b683-2328752ee600 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:10.973994 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 821/1654] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:10 2022] POST /identity/v3/projects => generated 405 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:10.978359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edbb72-1ef0-440f-b33b-3a0ae81cb2da admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:10.980184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 834/1655] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:11.114575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edbb72-1ef0-440f-b33b-3a0ae81cb2da admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:11.130039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f955d-be12-47e5-9e17-761da7faeed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:11.130785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f955d-be12-47e5-9e17-761da7faeed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:11.131074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f955d-be12-47e5-9e17-761da7faeed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:11.131888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f955d-be12-47e5-9e17-761da7faeed8 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:11.132606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f955d-be12-47e5-9e17-761da7faeed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:11.134609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f955d-be12-47e5-9e17-761da7faeed8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:11.159964 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 822/1656] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:10 2022] POST /identity/v3/users => generated 309 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:36:11.160830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f752a6-8596-4993-b683-2328752ee600 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:11.173514 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f752a6-8596-4993-b683-2328752ee600 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:11.188021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d33f2-5adb-4a60-9405-fa7ff4e88d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:11.189413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d33f2-5adb-4a60-9405-fa7ff4e88d1e admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:11.197208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d33f2-5adb-4a60-9405-fa7ff4e88d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:11.197862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d33f2-5adb-4a60-9405-fa7ff4e88d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:11.201509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d33f2-5adb-4a60-9405-fa7ff4e88d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:11.206906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d33f2-5adb-4a60-9405-fa7ff4e88d1e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:11.237934 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 835/1657] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:36:11.239336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f955d-be12-47e5-9e17-761da7faeed8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:11.254229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f955d-be12-47e5-9e17-761da7faeed8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:11.266241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62dabb-fb17-4e21-ae83-ff6af8d2b239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:11.267234 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62dabb-fb17-4e21-ae83-ff6af8d2b239 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:11.267733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62dabb-fb17-4e21-ae83-ff6af8d2b239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:11.268054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62dabb-fb17-4e21-ae83-ff6af8d2b239 admin admin] PATH_INFO: `/v3/projects/675398b7217441d4aa08ccb2ff1ddeeb/users/49856c7cf5644e8c80920595460039aa/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:11.298808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bae73-1c24-4672-b101-b602eab183d8 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:11.299315 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bae73-1c24-4672-b101-b602eab183d8 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:11.299889 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bae73-1c24-4672-b101-b602eab183d8 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:11.301713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 836/1658] 10.222.0.39 () {54 vars in 912 bytes} [Tue Jun 7 08:36:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:11.308387 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d4483-0634-462f-acd7-48adef6bc90b None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:11.308837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d4483-0634-462f-acd7-48adef6bc90b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:11.309800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d4483-0634-462f-acd7-48adef6bc90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:11.313663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62dabb-fb17-4e21-ae83-ff6af8d2b239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=675398b7217441d4aa08ccb2ff1ddeeb, user_id=49856c7cf5644e8c80920595460039aa, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:11.320711 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62dabb-fb17-4e21-ae83-ff6af8d2b239 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:11.344213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 823/1659] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:11 2022] PUT /identity/v3/projects/675398b7217441d4aa08ccb2ff1ddeeb/users/49856c7cf5644e8c80920595460039aa/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:11.350650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62dabb-fb17-4e21-ae83-ff6af8d2b239 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:11.371189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:11.449270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-3b9d4483-0634-462f-acd7-48adef6bc90b None None] MFA Rules not processed for user `7ddf89deb28749eda2d26ad66bc282f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:11.452654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-3b9d4483-0634-462f-acd7-48adef6bc90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:11.503429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62dabb-fb17-4e21-ae83-ff6af8d2b239 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:11.522493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0b407-11d5-4cbc-ba4a-e1f1e0d0a5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:11.522824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0b407-11d5-4cbc-ba4a-e1f1e0d0a5fd admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:11.523113 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0b407-11d5-4cbc-ba4a-e1f1e0d0a5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:11.523434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0b407-11d5-4cbc-ba4a-e1f1e0d0a5fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:11.524186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0b407-11d5-4cbc-ba4a-e1f1e0d0a5fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:11.525991 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0b407-11d5-4cbc-ba4a-e1f1e0d0a5fd admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:11.547604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 824/1660] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:11.549067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0b407-11d5-4cbc-ba4a-e1f1e0d0a5fd admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:11.580071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0b407-11d5-4cbc-ba4a-e1f1e0d0a5fd admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:11.589619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 837/1661] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jun 7 08:36:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:11.598962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3058ca6-58df-41dd-bfdb-1e17c072210a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:11.600874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3058ca6-58df-41dd-bfdb-1e17c072210a admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:11.600874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3058ca6-58df-41dd-bfdb-1e17c072210a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:11.601054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3058ca6-58df-41dd-bfdb-1e17c072210a admin admin] PATH_INFO: `/v3/projects/675398b7217441d4aa08ccb2ff1ddeeb/users/49856c7cf5644e8c80920595460039aa/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:11.610847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3058ca6-58df-41dd-bfdb-1e17c072210a admin admin] RBAC: Authorizing `identity:create_grant(project_id=675398b7217441d4aa08ccb2ff1ddeeb, user_id=49856c7cf5644e8c80920595460039aa, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:11.613773 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3058ca6-58df-41dd-bfdb-1e17c072210a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:11.638020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d4483-0634-462f-acd7-48adef6bc90b None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:11.653198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 825/1662] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:11 2022] PUT /identity/v3/projects/675398b7217441d4aa08ccb2ff1ddeeb/users/49856c7cf5644e8c80920595460039aa/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:11.660531 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d4483-0634-462f-acd7-48adef6bc90b None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:11.674149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:11.767590 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f308c-d00d-4bd4-b9b1-d615d3ba288d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:11.768271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f308c-d00d-4bd4-b9b1-d615d3ba288d service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:11.768552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f308c-d00d-4bd4-b9b1-d615d3ba288d service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:11.768829 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f308c-d00d-4bd4-b9b1-d615d3ba288d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:11.784954 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f308c-d00d-4bd4-b9b1-d615d3ba288d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:11.787467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f308c-d00d-4bd4-b9b1-d615d3ba288d service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:11.809870 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 838/1663] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:36:13.396536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3058ca6-58df-41dd-bfdb-1e17c072210a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:13.399510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f3058ca6-58df-41dd-bfdb-1e17c072210a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:13.655359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3058ca6-58df-41dd-bfdb-1e17c072210a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:13.670008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdd512-bc1a-43c0-8017-58aebb5cd21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:13.670993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdd512-bc1a-43c0-8017-58aebb5cd21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:13.671347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdd512-bc1a-43c0-8017-58aebb5cd21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:13.671700 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdd512-bc1a-43c0-8017-58aebb5cd21e admin admin] PATH_INFO: `/v3/projects/956d6e9aeb1c4279955c146130b1fb7b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:13.675894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdd512-bc1a-43c0-8017-58aebb5cd21e admin admin] RBAC: Authorizing `identity:delete_project(project_id=956d6e9aeb1c4279955c146130b1fb7b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:13.678026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdd512-bc1a-43c0-8017-58aebb5cd21e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:13.820314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-52fdd512-bc1a-43c0-8017-58aebb5cd21e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '956d6e9aeb1c4279955c146130b1fb7b', 'request_id': 'req-52fdd512-bc1a-43c0-8017-58aebb5cd21e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:13.850942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-52fdd512-bc1a-43c0-8017-58aebb5cd21e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '956d6e9aeb1c4279955c146130b1fb7b', 'request_id': 'req-52fdd512-bc1a-43c0-8017-58aebb5cd21e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:13.895231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-52fdd512-bc1a-43c0-8017-58aebb5cd21e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '956d6e9aeb1c4279955c146130b1fb7b', 'request_id': 'req-52fdd512-bc1a-43c0-8017-58aebb5cd21e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:13.927874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f308c-d00d-4bd4-b9b1-d615d3ba288d service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:13.944336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-52fdd512-bc1a-43c0-8017-58aebb5cd21e admin admin] The token cache is being invalidate because project 956d6e9aeb1c4279955c146130b1fb7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:13.945434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-52fdd512-bc1a-43c0-8017-58aebb5cd21e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:13.950022 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-52fdd512-bc1a-43c0-8017-58aebb5cd21e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:13.952086 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ed0f308c-d00d-4bd4-b9b1-d615d3ba288d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:13.955256 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 826/1664] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:13 2022] DELETE /identity/v3/projects/956d6e9aeb1c4279955c146130b1fb7b => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:13.962402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8bc06-02c1-43d9-aa0f-0ce00963334b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:13.962922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8bc06-02c1-43d9-aa0f-0ce00963334b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:13.963443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8bc06-02c1-43d9-aa0f-0ce00963334b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:13.995612 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f308c-d00d-4bd4-b9b1-d615d3ba288d service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:14.034686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c4c8bc06-02c1-43d9-aa0f-0ce00963334b None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:14.036242 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c4c8bc06-02c1-43d9-aa0f-0ce00963334b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:14.081561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:14.222311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 827/1665] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:14.224131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bddcd-fab7-436b-9d41-43a7b0df729a None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:14.228645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bddcd-fab7-436b-9d41-43a7b0df729a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:14.229245 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bddcd-fab7-436b-9d41-43a7b0df729a None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:14.231754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 828/1666] 10.222.0.39 () {54 vars in 960 bytes} [Tue Jun 7 08:36:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:14.233279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da2bde-841c-4740-ac01-a1c7b101d15e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:14.234475 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da2bde-841c-4740-ac01-a1c7b101d15e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:14.235498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da2bde-841c-4740-ac01-a1c7b101d15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:14.270115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:14.282253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:14.284862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:14.285331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] PATH_INFO: `/v3/projects/7cdf56452f2f49d792327ead98000779` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:14.285911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-64da2bde-841c-4740-ac01-a1c7b101d15e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:14.287296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-64da2bde-841c-4740-ac01-a1c7b101d15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:14.288255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cdf56452f2f49d792327ead98000779)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:14.290646 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: WARNING py.warnings [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:36:14.290646 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: warnings.warn(msg) Jun 07 08:36:14.290646 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: Jun 07 08:36:14.292858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:14.321071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 829/1667] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:14.322763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0845a70-a1ba-40b8-ae8b-240cf41d5eed None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:14.323062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0845a70-a1ba-40b8-ae8b-240cf41d5eed None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:14.323346 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0845a70-a1ba-40b8-ae8b-240cf41d5eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:14.377936 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e0845a70-a1ba-40b8-ae8b-240cf41d5eed None None] MFA Rules not processed for user `7a7d1e8a0f4b4511a7dc8b27651744c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:14.378900 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e0845a70-a1ba-40b8-ae8b-240cf41d5eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:14.392139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cdf56452f2f49d792327ead98000779', 'request_id': 'req-726a17eb-38de-48d0-86fc-c2d1469eb652'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:14.417912 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cdf56452f2f49d792327ead98000779', 'request_id': 'req-726a17eb-38de-48d0-86fc-c2d1469eb652'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:14.450868 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cdf56452f2f49d792327ead98000779', 'request_id': 'req-726a17eb-38de-48d0-86fc-c2d1469eb652'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:14.480650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] The token cache is being invalidate because project 7cdf56452f2f49d792327ead98000779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:14.482039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:14.482039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:14.485411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 839/1668] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:13 2022] DELETE /identity/v3/projects/7cdf56452f2f49d792327ead98000779 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:14.486407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:14.491398 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:14.552279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 830/1669] 10.222.0.39 () {58 vars in 1057 bytes} [Tue Jun 7 08:36:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:14.555931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0845a70-a1ba-40b8-ae8b-240cf41d5eed None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:14.559301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e0845a70-a1ba-40b8-ae8b-240cf41d5eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:14.612708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:14.679752 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a17eb-38de-48d0-86fc-c2d1469eb652 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:14.719230 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1221e8-2d9e-47ea-851a-2b0dc50bd701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:14.721898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1221e8-2d9e-47ea-851a-2b0dc50bd701 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:14.729491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1221e8-2d9e-47ea-851a-2b0dc50bd701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:14.729858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1221e8-2d9e-47ea-851a-2b0dc50bd701 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:14.730680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1221e8-2d9e-47ea-851a-2b0dc50bd701 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:14.733737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1221e8-2d9e-47ea-851a-2b0dc50bd701 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:14.739410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1221e8-2d9e-47ea-851a-2b0dc50bd701 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:14.778824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0845a70-a1ba-40b8-ae8b-240cf41d5eed None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:14.793026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 840/1670] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:14.793732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1221e8-2d9e-47ea-851a-2b0dc50bd701 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:14.825579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdea80-199d-480b-a690-e3437497b05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:14.827912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdea80-199d-480b-a690-e3437497b05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:14.828539 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdea80-199d-480b-a690-e3437497b05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:14.829372 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdea80-199d-480b-a690-e3437497b05a admin admin] PATH_INFO: `/v3/users/d195c70b52ec47e18ba224d90687905b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:14.833686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdea80-199d-480b-a690-e3437497b05a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d195c70b52ec47e18ba224d90687905b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:14.837397 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdea80-199d-480b-a690-e3437497b05a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:14.838745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1221e8-2d9e-47ea-851a-2b0dc50bd701 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:14.867091 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e55cff-7f6f-4ba2-8b8c-2a2d415dd82b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:14.868476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e55cff-7f6f-4ba2-8b8c-2a2d415dd82b service placement] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:14.872314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e55cff-7f6f-4ba2-8b8c-2a2d415dd82b service placement] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:14.872639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e55cff-7f6f-4ba2-8b8c-2a2d415dd82b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:14.882871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f9e55cff-7f6f-4ba2-8b8c-2a2d415dd82b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:14.936132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e55cff-7f6f-4ba2-8b8c-2a2d415dd82b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:14.937929 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e55cff-7f6f-4ba2-8b8c-2a2d415dd82b service placement] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:14.961128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d7bdea80-199d-480b-a690-e3437497b05a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd195c70b52ec47e18ba224d90687905b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:14.964320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d7bdea80-199d-480b-a690-e3437497b05a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd195c70b52ec47e18ba224d90687905b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:14.984617 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:14.990056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 841/1671] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:14.993039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e55cff-7f6f-4ba2-8b8c-2a2d415dd82b service placement] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:15.001351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d7bdea80-199d-480b-a690-e3437497b05a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd195c70b52ec47e18ba224d90687905b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:15.002190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d7bdea80-199d-480b-a690-e3437497b05a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd195c70b52ec47e18ba224d90687905b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:15.074571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e55cff-7f6f-4ba2-8b8c-2a2d415dd82b service placement] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:15.078078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d7bdea80-199d-480b-a690-e3437497b05a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd195c70b52ec47e18ba224d90687905b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:15.094089 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab34d0-db12-493f-9d7c-fe7c1a799ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:15.094789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab34d0-db12-493f-9d7c-fe7c1a799ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:15.095121 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab34d0-db12-493f-9d7c-fe7c1a799ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:15.095404 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab34d0-db12-493f-9d7c-fe7c1a799ba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:15.105289 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab34d0-db12-493f-9d7c-fe7c1a799ba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:15.105289 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab34d0-db12-493f-9d7c-fe7c1a799ba8 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:15.144954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 831/1672] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:14 2022] DELETE /identity/v3/users/d195c70b52ec47e18ba224d90687905b => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:15.156454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78c1ca-d8e8-456c-943e-5c7a9c93d388 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:15.156896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78c1ca-d8e8-456c-943e-5c7a9c93d388 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:15.157107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78c1ca-d8e8-456c-943e-5c7a9c93d388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:15.207960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 842/1673] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:14 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:15.212149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab34d0-db12-493f-9d7c-fe7c1a799ba8 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:15.214078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e9ab34d0-db12-493f-9d7c-fe7c1a799ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:15.249457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7f78c1ca-d8e8-456c-943e-5c7a9c93d388 None None] MFA Rules not processed for user `49856c7cf5644e8c80920595460039aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:15.254204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7f78c1ca-d8e8-456c-943e-5c7a9c93d388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:15.543615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab34d0-db12-493f-9d7c-fe7c1a799ba8 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:15.564706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096d3d4-e461-48b1-8359-d7d58d6b0a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:15.567521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096d3d4-e461-48b1-8359-d7d58d6b0a60 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:15.568003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096d3d4-e461-48b1-8359-d7d58d6b0a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:15.568676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096d3d4-e461-48b1-8359-d7d58d6b0a60 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:15.577464 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096d3d4-e461-48b1-8359-d7d58d6b0a60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:15.580057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096d3d4-e461-48b1-8359-d7d58d6b0a60 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:15.612686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 832/1674] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:15 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:15.614062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78c1ca-d8e8-456c-943e-5c7a9c93d388 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:15.617713 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7f78c1ca-d8e8-456c-943e-5c7a9c93d388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:15.651642 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 843/1675] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:15 2022] POST /identity/v3/users => generated 287 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:36:15.654706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096d3d4-e461-48b1-8359-d7d58d6b0a60 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:15.659927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78c1ca-d8e8-456c-943e-5c7a9c93d388 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:15.672741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73009255-931c-4ef3-8526-bf7264f35d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:15.673859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73009255-931c-4ef3-8526-bf7264f35d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:15.674144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73009255-931c-4ef3-8526-bf7264f35d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:15.674417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73009255-931c-4ef3-8526-bf7264f35d7c admin admin] PATH_INFO: `/v3/users/e1214914fade457d9027bb097911920c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:15.684287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73009255-931c-4ef3-8526-bf7264f35d7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1214914fade457d9027bb097911920c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:15.684630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096d3d4-e461-48b1-8359-d7d58d6b0a60 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:15.687756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73009255-931c-4ef3-8526-bf7264f35d7c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:15.698186 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a80ba-732a-43bc-b15e-647743256ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:15.699620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a80ba-732a-43bc-b15e-647743256ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:15.699917 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a80ba-732a-43bc-b15e-647743256ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:15.700198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a80ba-732a-43bc-b15e-647743256ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:15.700940 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a80ba-732a-43bc-b15e-647743256ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:15.709132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a80ba-732a-43bc-b15e-647743256ef8 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:15.723638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 844/1676] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:15.728678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a80ba-732a-43bc-b15e-647743256ef8 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:15.754020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a80ba-732a-43bc-b15e-647743256ef8 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:15.766979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe09d0f-c5f4-4d1f-b661-602db6aeda38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:15.768303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe09d0f-c5f4-4d1f-b661-602db6aeda38 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:15.768933 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe09d0f-c5f4-4d1f-b661-602db6aeda38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:15.769064 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe09d0f-c5f4-4d1f-b661-602db6aeda38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:15.770218 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe09d0f-c5f4-4d1f-b661-602db6aeda38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:15.773676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-73009255-931c-4ef3-8526-bf7264f35d7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1214914fade457d9027bb097911920c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:15.775123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-73009255-931c-4ef3-8526-bf7264f35d7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1214914fade457d9027bb097911920c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:15.777384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe09d0f-c5f4-4d1f-b661-602db6aeda38 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:15.795042 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-73009255-931c-4ef3-8526-bf7264f35d7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1214914fade457d9027bb097911920c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:15.795865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-73009255-931c-4ef3-8526-bf7264f35d7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1214914fade457d9027bb097911920c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:15.799799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 845/1677] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:15.801006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe09d0f-c5f4-4d1f-b661-602db6aeda38 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:15.805421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-cfe09d0f-c5f4-4d1f-b661-602db6aeda38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:15.820885 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-73009255-931c-4ef3-8526-bf7264f35d7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1214914fade457d9027bb097911920c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:15.821656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe09d0f-c5f4-4d1f-b661-602db6aeda38 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:15.827894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d80a0-759b-496b-92e1-6ca2a3276b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:15.828807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d80a0-759b-496b-92e1-6ca2a3276b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:15.829112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d80a0-759b-496b-92e1-6ca2a3276b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:15.829402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d80a0-759b-496b-92e1-6ca2a3276b83 admin admin] PATH_INFO: `/v3/projects/e581913bddbe46e38355b9f9bbee3243/users/84f2eb715c1940e89ee1127926e44751/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:15.855584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 833/1678] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:15 2022] DELETE /identity/v3/users/e1214914fade457d9027bb097911920c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:15.857819 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d80a0-759b-496b-92e1-6ca2a3276b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e581913bddbe46e38355b9f9bbee3243, user_id=84f2eb715c1940e89ee1127926e44751, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:15.859341 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737f131-e366-4add-97a6-3a57cba7e931 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:15.859738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737f131-e366-4add-97a6-3a57cba7e931 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:15.860142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737f131-e366-4add-97a6-3a57cba7e931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:15.861456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d80a0-759b-496b-92e1-6ca2a3276b83 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:15.909820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 846/1679] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:15 2022] PUT /identity/v3/projects/e581913bddbe46e38355b9f9bbee3243/users/84f2eb715c1940e89ee1127926e44751/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:15.910954 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56445d6c-8d7a-49ce-94ed-6cdde54952f8 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:15.911437 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56445d6c-8d7a-49ce-94ed-6cdde54952f8 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:15.911719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56445d6c-8d7a-49ce-94ed-6cdde54952f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:15.925709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3737f131-e366-4add-97a6-3a57cba7e931 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:15.934627 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3737f131-e366-4add-97a6-3a57cba7e931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:15.971785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:15.997912 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-56445d6c-8d7a-49ce-94ed-6cdde54952f8 None None] MFA Rules not processed for user `84f2eb715c1940e89ee1127926e44751`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:15.999170 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-56445d6c-8d7a-49ce-94ed-6cdde54952f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:16.122919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 834/1680] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:16.133763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:16.180874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 847/1681] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:16.184389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737f131-e366-4add-97a6-3a57cba7e931 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:16.198309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737f131-e366-4add-97a6-3a57cba7e931 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:16.203620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45cf93-823b-411d-b688-be72c914e9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:16.204774 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45cf93-823b-411d-b688-be72c914e9e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:16.205980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45cf93-823b-411d-b688-be72c914e9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:16.206452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45cf93-823b-411d-b688-be72c914e9e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:16.207361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45cf93-823b-411d-b688-be72c914e9e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:16.210125 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45cf93-823b-411d-b688-be72c914e9e6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:16.245069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56445d6c-8d7a-49ce-94ed-6cdde54952f8 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:16.246952 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-56445d6c-8d7a-49ce-94ed-6cdde54952f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:16.269779 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 835/1682] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:16 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:16.273388 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45cf93-823b-411d-b688-be72c914e9e6 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:16.465220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56445d6c-8d7a-49ce-94ed-6cdde54952f8 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:16.547250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45cf93-823b-411d-b688-be72c914e9e6 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:16.580328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99701f74-09d3-4b33-8cf5-6f1b564c78a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:16.587895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99701f74-09d3-4b33-8cf5-6f1b564c78a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:16.588520 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99701f74-09d3-4b33-8cf5-6f1b564c78a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:16.592149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99701f74-09d3-4b33-8cf5-6f1b564c78a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:16.594537 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99701f74-09d3-4b33-8cf5-6f1b564c78a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:16.607709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99701f74-09d3-4b33-8cf5-6f1b564c78a8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:16.681421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa9310-973a-4a6d-9b18-2052d9fcbfa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:16.682132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa9310-973a-4a6d-9b18-2052d9fcbfa1 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:16.682334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa9310-973a-4a6d-9b18-2052d9fcbfa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:16.682624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa9310-973a-4a6d-9b18-2052d9fcbfa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:16.687852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 836/1683] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:16 2022] POST /identity/v3/users => generated 293 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:16.688991 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99701f74-09d3-4b33-8cf5-6f1b564c78a8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:16.697769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa9310-973a-4a6d-9b18-2052d9fcbfa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:16.700822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa9310-973a-4a6d-9b18-2052d9fcbfa1 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:16.719016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 848/1684] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:16.733984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99701f74-09d3-4b33-8cf5-6f1b564c78a8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:16.752705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f867580-f441-4bfd-ba99-eb2662e749d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:16.753593 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f867580-f441-4bfd-ba99-eb2662e749d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:16.753880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f867580-f441-4bfd-ba99-eb2662e749d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:16.754162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f867580-f441-4bfd-ba99-eb2662e749d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:16.754927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f867580-f441-4bfd-ba99-eb2662e749d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:16.760144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f867580-f441-4bfd-ba99-eb2662e749d1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:16.795005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa9310-973a-4a6d-9b18-2052d9fcbfa1 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:16.796445 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 837/1685] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:16.813402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa9310-973a-4a6d-9b18-2052d9fcbfa1 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:16.826302 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e92aef-01b9-4c7c-accc-dc42e9254e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:16.827247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e92aef-01b9-4c7c-accc-dc42e9254e5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:16.827540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e92aef-01b9-4c7c-accc-dc42e9254e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:16.827825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e92aef-01b9-4c7c-accc-dc42e9254e5a admin admin] PATH_INFO: `/v3/projects/2eafbcdcfa404458ac0a0472ef54c27c/users/4b42d2d116ec4591af8cc120ce2f9881/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:16.875269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e92aef-01b9-4c7c-accc-dc42e9254e5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eafbcdcfa404458ac0a0472ef54c27c, user_id=4b42d2d116ec4591af8cc120ce2f9881, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:16.878082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e92aef-01b9-4c7c-accc-dc42e9254e5a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:16.919410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 849/1686] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:16 2022] PUT /identity/v3/projects/2eafbcdcfa404458ac0a0472ef54c27c/users/4b42d2d116ec4591af8cc120ce2f9881/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:36:16.922895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f867580-f441-4bfd-ba99-eb2662e749d1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:16.932161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e92aef-01b9-4c7c-accc-dc42e9254e5a admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:16.934140 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-99e92aef-01b9-4c7c-accc-dc42e9254e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:16.945725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f867580-f441-4bfd-ba99-eb2662e749d1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:17.111776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e92aef-01b9-4c7c-accc-dc42e9254e5a admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:17.119260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f70aba-f1a6-4b94-a3bd-dc96b476229a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:17.119959 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f70aba-f1a6-4b94-a3bd-dc96b476229a admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:17.120242 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f70aba-f1a6-4b94-a3bd-dc96b476229a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:17.120517 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f70aba-f1a6-4b94-a3bd-dc96b476229a admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:17.121427 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f70aba-f1a6-4b94-a3bd-dc96b476229a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:17.125094 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f70aba-f1a6-4b94-a3bd-dc96b476229a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:17.125652 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eee5e6-4a9d-4509-804c-53915540b8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:17.126639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eee5e6-4a9d-4509-804c-53915540b8ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:17.126818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eee5e6-4a9d-4509-804c-53915540b8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:17.127100 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eee5e6-4a9d-4509-804c-53915540b8ec admin admin] PATH_INFO: `/v3/projects/54b04e59a81a4a659dafabbebaf44b50` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:17.128368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eee5e6-4a9d-4509-804c-53915540b8ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=54b04e59a81a4a659dafabbebaf44b50)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:17.130388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eee5e6-4a9d-4509-804c-53915540b8ec admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:17.160854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:17.169660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 838/1687] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:17.170613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f70aba-f1a6-4b94-a3bd-dc96b476229a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:17.201040 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f70aba-f1a6-4b94-a3bd-dc96b476229a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:17.208593 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e5b67-2044-4f43-8f10-1c0a30e72691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:17.209590 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e5b67-2044-4f43-8f10-1c0a30e72691 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:17.210123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e5b67-2044-4f43-8f10-1c0a30e72691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:17.210482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e5b67-2044-4f43-8f10-1c0a30e72691 admin admin] PATH_INFO: `/v3/projects/2eafbcdcfa404458ac0a0472ef54c27c/users/4b42d2d116ec4591af8cc120ce2f9881/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:17.214209 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e5b67-2044-4f43-8f10-1c0a30e72691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eafbcdcfa404458ac0a0472ef54c27c, user_id=4b42d2d116ec4591af8cc120ce2f9881, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:17.223793 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e5b67-2044-4f43-8f10-1c0a30e72691 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:17.250967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-08eee5e6-4a9d-4509-804c-53915540b8ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54b04e59a81a4a659dafabbebaf44b50', 'request_id': 'req-08eee5e6-4a9d-4509-804c-53915540b8ec'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:17.264308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 839/1688] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:17 2022] PUT /identity/v3/projects/2eafbcdcfa404458ac0a0472ef54c27c/users/4b42d2d116ec4591af8cc120ce2f9881/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:17.264994 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190b226-8948-4f27-8395-92a8ee4ade25 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:17.265315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190b226-8948-4f27-8395-92a8ee4ade25 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:17.265623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190b226-8948-4f27-8395-92a8ee4ade25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:17.266195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-08eee5e6-4a9d-4509-804c-53915540b8ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54b04e59a81a4a659dafabbebaf44b50', 'request_id': 'req-08eee5e6-4a9d-4509-804c-53915540b8ec'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:17.285867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-08eee5e6-4a9d-4509-804c-53915540b8ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54b04e59a81a4a659dafabbebaf44b50', 'request_id': 'req-08eee5e6-4a9d-4509-804c-53915540b8ec'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:17.286100 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:17.309098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-08eee5e6-4a9d-4509-804c-53915540b8ec admin admin] The token cache is being invalidate because project 54b04e59a81a4a659dafabbebaf44b50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:17.309407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-08eee5e6-4a9d-4509-804c-53915540b8ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:17.312627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-08eee5e6-4a9d-4509-804c-53915540b8ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:17.320988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 850/1689] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:16 2022] DELETE /identity/v3/projects/54b04e59a81a4a659dafabbebaf44b50 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:17.323179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f784a-0dba-4417-ba1a-8af9372ead58 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:17.323472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f784a-0dba-4417-ba1a-8af9372ead58 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:17.323790 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f784a-0dba-4417-ba1a-8af9372ead58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:17.354554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f190b226-8948-4f27-8395-92a8ee4ade25 None None] MFA Rules not processed for user `4b42d2d116ec4591af8cc120ce2f9881`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:17.358274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f190b226-8948-4f27-8395-92a8ee4ade25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:17.373482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-a34f784a-0dba-4417-ba1a-8af9372ead58 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:17.375816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a34f784a-0dba-4417-ba1a-8af9372ead58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:17.499246 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 851/1690] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:17.507398 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf613d-6113-4817-910e-66051704835d None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:17.507804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf613d-6113-4817-910e-66051704835d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:17.509213 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf613d-6113-4817-910e-66051704835d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:17.541781 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 840/1691] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:17.551551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190b226-8948-4f27-8395-92a8ee4ade25 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:17.552208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-71bf613d-6113-4817-910e-66051704835d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:17.553909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f190b226-8948-4f27-8395-92a8ee4ade25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:17.554320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-71bf613d-6113-4817-910e-66051704835d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:17.578464 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 852/1692] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:17.580621 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf613d-6113-4817-910e-66051704835d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:17.612181 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf613d-6113-4817-910e-66051704835d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:17.624620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de744fc-c743-406b-905b-dc0aa0f075f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:17.625948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de744fc-c743-406b-905b-dc0aa0f075f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:17.626263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de744fc-c743-406b-905b-dc0aa0f075f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:17.626545 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de744fc-c743-406b-905b-dc0aa0f075f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:17.629186 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de744fc-c743-406b-905b-dc0aa0f075f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:17.629507 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de744fc-c743-406b-905b-dc0aa0f075f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:17.635398 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de744fc-c743-406b-905b-dc0aa0f075f7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:17.643657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190b226-8948-4f27-8395-92a8ee4ade25 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:17.669189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 853/1693] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:17.674110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de744fc-c743-406b-905b-dc0aa0f075f7 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:17.676976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb7ba8-5542-419e-9d67-bac551b8c8a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:17.681507 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7ba8-5542-419e-9d67-bac551b8c8a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:17.683136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7ba8-5542-419e-9d67-bac551b8c8a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:17.684030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7ba8-5542-419e-9d67-bac551b8c8a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:17.700781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de744fc-c743-406b-905b-dc0aa0f075f7 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:17.712041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d42730-cb36-4877-8f2f-1c58b012dee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:17.714872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d42730-cb36-4877-8f2f-1c58b012dee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:17.715187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d42730-cb36-4877-8f2f-1c58b012dee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:17.715571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d42730-cb36-4877-8f2f-1c58b012dee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:17.717979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d42730-cb36-4877-8f2f-1c58b012dee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:17.722300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d42730-cb36-4877-8f2f-1c58b012dee7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:17.723635 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb7ba8-5542-419e-9d67-bac551b8c8a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:17.727699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb7ba8-5542-419e-9d67-bac551b8c8a5 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:17.781917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 841/1694] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:36:17.788897 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 854/1695] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:17 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:17.793343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb7ba8-5542-419e-9d67-bac551b8c8a5 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:17.827814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb7ba8-5542-419e-9d67-bac551b8c8a5 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:18.067438 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75110e99-601d-4802-9629-64d96f62db13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:18.069539 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75110e99-601d-4802-9629-64d96f62db13 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:18.070366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75110e99-601d-4802-9629-64d96f62db13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:18.071098 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75110e99-601d-4802-9629-64d96f62db13 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:18.073771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75110e99-601d-4802-9629-64d96f62db13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:18.079754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75110e99-601d-4802-9629-64d96f62db13 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:18.142469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 842/1696] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:17 2022] POST /identity/v3/users => generated 287 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:18.149294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d42730-cb36-4877-8f2f-1c58b012dee7 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:18.166740 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d42730-cb36-4877-8f2f-1c58b012dee7 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:18.177928 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64a633-68c9-4305-a45e-acb8b238bcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:18.180968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64a633-68c9-4305-a45e-acb8b238bcd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:18.181560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64a633-68c9-4305-a45e-acb8b238bcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:18.181803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64a633-68c9-4305-a45e-acb8b238bcd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:18.182619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64a633-68c9-4305-a45e-acb8b238bcd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:18.187722 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64a633-68c9-4305-a45e-acb8b238bcd6 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:18.211578 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 855/1697] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:18.215651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75110e99-601d-4802-9629-64d96f62db13 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:18.234068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75110e99-601d-4802-9629-64d96f62db13 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:18.246950 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e9fef-9047-4e3f-a9d2-da2b64501de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:18.247699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e9fef-9047-4e3f-a9d2-da2b64501de7 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:18.247985 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e9fef-9047-4e3f-a9d2-da2b64501de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:18.248263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e9fef-9047-4e3f-a9d2-da2b64501de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:18.249041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e9fef-9047-4e3f-a9d2-da2b64501de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:18.250683 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e9fef-9047-4e3f-a9d2-da2b64501de7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:18.268541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64a633-68c9-4305-a45e-acb8b238bcd6 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:18.278353 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 843/1698] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:18.285652 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a7479-a32a-47b5-acb5-dc2396494e5e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:18.286077 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a7479-a32a-47b5-acb5-dc2396494e5e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:18.286539 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a7479-a32a-47b5-acb5-dc2396494e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:18.290015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64a633-68c9-4305-a45e-acb8b238bcd6 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:18.305743 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d26ac-9616-4def-b3db-6ea8ab32295d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:18.307520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d26ac-9616-4def-b3db-6ea8ab32295d admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:18.326515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d26ac-9616-4def-b3db-6ea8ab32295d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:18.327043 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d26ac-9616-4def-b3db-6ea8ab32295d admin admin] PATH_INFO: `/v3/projects/21545f05eda441d1b440788e5fbfc880/users/6a3bb7f368df40419b070182ea60a006/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:18.375831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-964a7479-a32a-47b5-acb5-dc2396494e5e None None] MFA Rules not processed for user `26d6eb06b23b4db88261ae58d650736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:18.379318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-964a7479-a32a-47b5-acb5-dc2396494e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:18.388247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d26ac-9616-4def-b3db-6ea8ab32295d admin admin] RBAC: Authorizing `identity:create_grant(project_id=21545f05eda441d1b440788e5fbfc880, user_id=6a3bb7f368df40419b070182ea60a006, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:18.394392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d26ac-9616-4def-b3db-6ea8ab32295d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:18.395706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:18.445796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 856/1699] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:18 2022] PUT /identity/v3/projects/21545f05eda441d1b440788e5fbfc880/users/6a3bb7f368df40419b070182ea60a006/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:18.447800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac4ee8-c5a3-483a-a2ca-567054bb0346 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:18.448128 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac4ee8-c5a3-483a-a2ca-567054bb0346 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:18.448495 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac4ee8-c5a3-483a-a2ca-567054bb0346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:18.492526 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-46ac4ee8-c5a3-483a-a2ca-567054bb0346 None None] MFA Rules not processed for user `6a3bb7f368df40419b070182ea60a006`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:18.494353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-46ac4ee8-c5a3-483a-a2ca-567054bb0346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:18.530132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:18.575891 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 844/1700] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jun 7 08:36:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:18.683596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 857/1701] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:18.689487 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a7479-a32a-47b5-acb5-dc2396494e5e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:18.699741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac4ee8-c5a3-483a-a2ca-567054bb0346 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:18.701484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-46ac4ee8-c5a3-483a-a2ca-567054bb0346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:18.708041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a7479-a32a-47b5-acb5-dc2396494e5e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:18.791720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cb88e-40e6-4626-9073-203aed6aa44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:18.792711 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cb88e-40e6-4626-9073-203aed6aa44d admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:18.793045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cb88e-40e6-4626-9073-203aed6aa44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:18.793402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cb88e-40e6-4626-9073-203aed6aa44d admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:18.794246 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953cb88e-40e6-4626-9073-203aed6aa44d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:18.797564 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953cb88e-40e6-4626-9073-203aed6aa44d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:18.832143 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 845/1702] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:18 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:18.837918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cb88e-40e6-4626-9073-203aed6aa44d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:18.864562 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac4ee8-c5a3-483a-a2ca-567054bb0346 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:18.984465 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f34706-f3df-4189-b015-b5fccd59bc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:18.988809 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f34706-f3df-4189-b015-b5fccd59bc58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:18.989505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f34706-f3df-4189-b015-b5fccd59bc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:18.989505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f34706-f3df-4189-b015-b5fccd59bc58 admin admin] PATH_INFO: `/v3/users/84f2eb715c1940e89ee1127926e44751` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:18.993298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f34706-f3df-4189-b015-b5fccd59bc58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f2eb715c1940e89ee1127926e44751)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:18.995712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f34706-f3df-4189-b015-b5fccd59bc58 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:19.012932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cb88e-40e6-4626-9073-203aed6aa44d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:19.027918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ca4fd-95da-49ad-b916-b5ecc95ec890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:19.028764 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ca4fd-95da-49ad-b916-b5ecc95ec890 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:19.029178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ca4fd-95da-49ad-b916-b5ecc95ec890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:19.029446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ca4fd-95da-49ad-b916-b5ecc95ec890 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:19.030159 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ca4fd-95da-49ad-b916-b5ecc95ec890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:19.032268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ca4fd-95da-49ad-b916-b5ecc95ec890 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:19.076773 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 846/1703] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:18 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:19.081418 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ca4fd-95da-49ad-b916-b5ecc95ec890 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:19.081884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-41f34706-f3df-4189-b015-b5fccd59bc58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f2eb715c1940e89ee1127926e44751'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:19.083456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-41f34706-f3df-4189-b015-b5fccd59bc58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f2eb715c1940e89ee1127926e44751'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:19.092191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-938ca4fd-95da-49ad-b916-b5ecc95ec890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:19.092894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-41f34706-f3df-4189-b015-b5fccd59bc58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f2eb715c1940e89ee1127926e44751'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:19.099721 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:19.110661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-41f34706-f3df-4189-b015-b5fccd59bc58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f2eb715c1940e89ee1127926e44751'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:19.111685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-41f34706-f3df-4189-b015-b5fccd59bc58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f2eb715c1940e89ee1127926e44751'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:19.138377 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 858/1704] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:18 2022] DELETE /identity/v3/users/84f2eb715c1940e89ee1127926e44751 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:19.140994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721dcba-162b-4b7f-80ec-7676aad1a93f None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:19.141521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721dcba-162b-4b7f-80ec-7676aad1a93f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:19.146045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721dcba-162b-4b7f-80ec-7676aad1a93f None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:19.164347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 859/1705] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jun 7 08:36:19 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:19.172500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721dcba-162b-4b7f-80ec-7676aad1a93f None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:19.174661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a721dcba-162b-4b7f-80ec-7676aad1a93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:19.185024 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:19.290949 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ca4fd-95da-49ad-b916-b5ecc95ec890 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:19.326179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721dcba-162b-4b7f-80ec-7676aad1a93f None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:19.333670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb4a21-5b8c-4ac6-8e0d-5be4eca6f484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:19.335238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb4a21-5b8c-4ac6-8e0d-5be4eca6f484 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:19.336551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb4a21-5b8c-4ac6-8e0d-5be4eca6f484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:19.337347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb4a21-5b8c-4ac6-8e0d-5be4eca6f484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:19.339192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb4a21-5b8c-4ac6-8e0d-5be4eca6f484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:19.341998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb4a21-5b8c-4ac6-8e0d-5be4eca6f484 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:19.390724 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 860/1706] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:19.397617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc5696-7f53-426d-b506-80eade0ccced None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:19.398903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc5696-7f53-426d-b506-80eade0ccced None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:19.399836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc5696-7f53-426d-b506-80eade0ccced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:19.448897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06787c12-1880-47af-8cd9-aa2833f9a5a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586ddffa77a54521b42b2aa50f16cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:19.449750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06787c12-1880-47af-8cd9-aa2833f9a5a1 service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:19.450033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06787c12-1880-47af-8cd9-aa2833f9a5a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:19.450308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06787c12-1880-47af-8cd9-aa2833f9a5a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:19.453304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-06787c12-1880-47af-8cd9-aa2833f9a5a1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:19.515929 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-01cc5696-7f53-426d-b506-80eade0ccced None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:19.518751 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-01cc5696-7f53-426d-b506-80eade0ccced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:19.534222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 861/1707] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:19.536045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe8c3-84d3-4334-a86e-37ab88e654ae None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:19.536894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe8c3-84d3-4334-a86e-37ab88e654ae None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:19.538020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe8c3-84d3-4334-a86e-37ab88e654ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:19.564321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-8dafe8c3-84d3-4334-a86e-37ab88e654ae None None] MFA Rules not processed for user `7a7d1e8a0f4b4511a7dc8b27651744c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:19.566314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8dafe8c3-84d3-4334-a86e-37ab88e654ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:19.619320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06787c12-1880-47af-8cd9-aa2833f9a5a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:19.621325 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06787c12-1880-47af-8cd9-aa2833f9a5a1 service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:19.635821 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 847/1708] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jun 7 08:36:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:36:19.636903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06787c12-1880-47af-8cd9-aa2833f9a5a1 service swift] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:19.651679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06787c12-1880-47af-8cd9-aa2833f9a5a1 service swift] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:19.658831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7721247-ef16-4935-a2e1-804ae3441f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:19.660191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7721247-ef16-4935-a2e1-804ae3441f05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:19.660544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7721247-ef16-4935-a2e1-804ae3441f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:19.660893 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7721247-ef16-4935-a2e1-804ae3441f05 admin admin] PATH_INFO: `/v3/projects/9dd8ab6ef5fb4b81a37174d88a24de36/users/3205adc3df4246ad8f2eedc1df218ede/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:19.689356 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7721247-ef16-4935-a2e1-804ae3441f05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd8ab6ef5fb4b81a37174d88a24de36, user_id=3205adc3df4246ad8f2eedc1df218ede, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:19.697649 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7721247-ef16-4935-a2e1-804ae3441f05 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:19.726076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 848/1709] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:19 2022] PUT /identity/v3/projects/9dd8ab6ef5fb4b81a37174d88a24de36/users/3205adc3df4246ad8f2eedc1df218ede/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:19.727643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7721247-ef16-4935-a2e1-804ae3441f05 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:19.732621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b7721247-ef16-4935-a2e1-804ae3441f05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:19.800537 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 862/1710] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jun 7 08:36:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:19.805322 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafe8c3-84d3-4334-a86e-37ab88e654ae None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:19.858346 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafe8c3-84d3-4334-a86e-37ab88e654ae None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:20.056299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7721247-ef16-4935-a2e1-804ae3441f05 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:20.088611 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a6ea1-1dfe-4f9a-803c-6f546155e629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:20.088963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a6ea1-1dfe-4f9a-803c-6f546155e629 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:20.089400 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a6ea1-1dfe-4f9a-803c-6f546155e629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:20.089570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a6ea1-1dfe-4f9a-803c-6f546155e629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:20.121790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:20.165405 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01daa6-929c-41b0-94cd-30e64a27b83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:20.167297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01daa6-929c-41b0-94cd-30e64a27b83f admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:20.167586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01daa6-929c-41b0-94cd-30e64a27b83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:20.167873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01daa6-929c-41b0-94cd-30e64a27b83f admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:20.172548 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01daa6-929c-41b0-94cd-30e64a27b83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:20.177327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01daa6-929c-41b0-94cd-30e64a27b83f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:20.229223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 863/1711] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:20.231527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01daa6-929c-41b0-94cd-30e64a27b83f admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:20.240545 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4e01daa6-929c-41b0-94cd-30e64a27b83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:20.275514 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:20.306109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01daa6-929c-41b0-94cd-30e64a27b83f admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:20.330525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:20.337878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:20.340656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:20.341219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] PATH_INFO: `/v3/projects/ba94c7ecfaf447c79c89ad09c446c522` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:20.347418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba94c7ecfaf447c79c89ad09c446c522)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:20.359909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:20.394667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a6ea1-1dfe-4f9a-803c-6f546155e629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:20.399279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a6ea1-1dfe-4f9a-803c-6f546155e629 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:20.445624 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 849/1712] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:20.448627 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a6ea1-1dfe-4f9a-803c-6f546155e629 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:20.455069 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-869a6ea1-1dfe-4f9a-803c-6f546155e629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:20.547116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba94c7ecfaf447c79c89ad09c446c522', 'request_id': 'req-a9aa5904-8f01-423d-825b-f51137c808a7'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:20.567884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba94c7ecfaf447c79c89ad09c446c522', 'request_id': 'req-a9aa5904-8f01-423d-825b-f51137c808a7'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:20.581109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba94c7ecfaf447c79c89ad09c446c522', 'request_id': 'req-a9aa5904-8f01-423d-825b-f51137c808a7'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:20.615058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] The token cache is being invalidate because project ba94c7ecfaf447c79c89ad09c446c522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:20.615482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:20.618451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:20.620420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 864/1713] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:20 2022] DELETE /identity/v3/projects/ba94c7ecfaf447c79c89ad09c446c522 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:20.624656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:20.628708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:20.752385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a6ea1-1dfe-4f9a-803c-6f546155e629 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:20.813817 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa5904-8f01-423d-825b-f51137c808a7 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:20.829300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e3640-8d49-41dd-9702-80ea73629c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:20.830702 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e3640-8d49-41dd-9702-80ea73629c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:20.831086 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e3640-8d49-41dd-9702-80ea73629c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:20.831431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e3640-8d49-41dd-9702-80ea73629c53 admin admin] PATH_INFO: `/v3/projects/9dd8ab6ef5fb4b81a37174d88a24de36/users/3205adc3df4246ad8f2eedc1df218ede/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:20.844348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e3640-8d49-41dd-9702-80ea73629c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd8ab6ef5fb4b81a37174d88a24de36, user_id=3205adc3df4246ad8f2eedc1df218ede, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:20.852731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e3640-8d49-41dd-9702-80ea73629c53 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:20.901584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 865/1714] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:20 2022] PUT /identity/v3/projects/9dd8ab6ef5fb4b81a37174d88a24de36/users/3205adc3df4246ad8f2eedc1df218ede/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:20.901584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e3640-8d49-41dd-9702-80ea73629c53 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:20.901584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4d9e3640-8d49-41dd-9702-80ea73629c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:21.149968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e3640-8d49-41dd-9702-80ea73629c53 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:21.172755 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:21.173940 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:21.176957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:21.177340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] PATH_INFO: `/v3/users/ba17e70e69b2460bb0bdb9383e482ec6` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:21.183734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba17e70e69b2460bb0bdb9383e482ec6)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:21.192343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:21.202401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265219e-97af-487c-a363-3b9210bd5082 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:21.204723 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265219e-97af-487c-a363-3b9210bd5082 service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:21.205788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265219e-97af-487c-a363-3b9210bd5082 service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:21.206786 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265219e-97af-487c-a363-3b9210bd5082 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:21.211551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7265219e-97af-487c-a363-3b9210bd5082 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:21.228584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265219e-97af-487c-a363-3b9210bd5082 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:21.231042 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265219e-97af-487c-a363-3b9210bd5082 service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:21.247080 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 850/1715] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 801 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:21.255677 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1fec4-ac29-4840-a97b-148c3769e180 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:21.256723 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1fec4-ac29-4840-a97b-148c3769e180 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:21.257496 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1fec4-ac29-4840-a97b-148c3769e180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:21.322300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:21.322562 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-52b1fec4-ac29-4840-a97b-148c3769e180 None None] MFA Rules not processed for user `3205adc3df4246ad8f2eedc1df218ede`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:21.329661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-52b1fec4-ac29-4840-a97b-148c3769e180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:21.415691 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba17e70e69b2460bb0bdb9383e482ec6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:21.416397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba17e70e69b2460bb0bdb9383e482ec6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:21.454196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba17e70e69b2460bb0bdb9383e482ec6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:21.511421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba17e70e69b2460bb0bdb9383e482ec6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:21.512654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba17e70e69b2460bb0bdb9383e482ec6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:21.515036 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:21.593305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 866/1716] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:20 2022] DELETE /identity/v3/users/ba17e70e69b2460bb0bdb9383e482ec6 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:21.720467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 851/1717] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:21 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:21.737022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:21.739581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:21.999571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a4b3f-3d03-4bf4-a283-9d3982ce32b1 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:22.014195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed6ae6-32ec-41eb-952e-0c737e9b2942 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:22.014911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed6ae6-32ec-41eb-952e-0c737e9b2942 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:22.015197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed6ae6-32ec-41eb-952e-0c737e9b2942 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:22.015479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed6ae6-32ec-41eb-952e-0c737e9b2942 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:22.242462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed6ae6-32ec-41eb-952e-0c737e9b2942 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:22.245974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed6ae6-32ec-41eb-952e-0c737e9b2942 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:22.296354 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 867/1718] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:21 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:22.483150 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1fec4-ac29-4840-a97b-148c3769e180 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:22.626325 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1fec4-ac29-4840-a97b-148c3769e180 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:22.637910 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d85371-7b61-4a74-976d-c18fdcad3341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:22.638751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d85371-7b61-4a74-976d-c18fdcad3341 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:22.639038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d85371-7b61-4a74-976d-c18fdcad3341 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:22.639384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d85371-7b61-4a74-976d-c18fdcad3341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:22.642454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-12d85371-7b61-4a74-976d-c18fdcad3341 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:22.855701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d85371-7b61-4a74-976d-c18fdcad3341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:22.858633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d85371-7b61-4a74-976d-c18fdcad3341 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:22.919948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 852/1719] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:23.213650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094daef4-53ce-47ab-bb16-c90ca161d603 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:23.214577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094daef4-53ce-47ab-bb16-c90ca161d603 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:23.215618 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094daef4-53ce-47ab-bb16-c90ca161d603 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:23.218519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 868/1720] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jun 7 08:36:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:36:23.668388 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138403d-2ffb-4dca-851e-eb765235fdea None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:23.669792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138403d-2ffb-4dca-851e-eb765235fdea None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:23.670979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138403d-2ffb-4dca-851e-eb765235fdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:23.699150 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6138403d-2ffb-4dca-851e-eb765235fdea None None] MFA Rules not processed for user `7a7d1e8a0f4b4511a7dc8b27651744c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:23.701346 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6138403d-2ffb-4dca-851e-eb765235fdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:23.814051 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094daef4-53ce-47ab-bb16-c90ca161d603 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:23.816700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-094daef4-53ce-47ab-bb16-c90ca161d603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:23.993373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 853/1721] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jun 7 08:36:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:24.013785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd92f10-5942-4f85-b2af-18b0a69ad054 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:24.014087 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd92f10-5942-4f85-b2af-18b0a69ad054 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:24.025680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd92f10-5942-4f85-b2af-18b0a69ad054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:24.060367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:24.097806 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094daef4-53ce-47ab-bb16-c90ca161d603 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:24.099279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:24.113551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:24.114903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:24.115196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:24.115578 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] PATH_INFO: `/v3/projects/e581913bddbe46e38355b9f9bbee3243` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:24.117401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e581913bddbe46e38355b9f9bbee3243)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:24.121896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:24.127535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2cd92f10-5942-4f85-b2af-18b0a69ad054 None None] MFA Rules not processed for user `4f603b289d964ee3ba3931948acf6788`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:24.129038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2cd92f10-5942-4f85-b2af-18b0a69ad054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:24.305315 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e581913bddbe46e38355b9f9bbee3243', 'request_id': 'req-8fb65fe2-165b-477e-8f7a-5714f177c827'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:24.324433 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e581913bddbe46e38355b9f9bbee3243', 'request_id': 'req-8fb65fe2-165b-477e-8f7a-5714f177c827'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:24.345785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e581913bddbe46e38355b9f9bbee3243', 'request_id': 'req-8fb65fe2-165b-477e-8f7a-5714f177c827'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:24.389703 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] The token cache is being invalidate because project e581913bddbe46e38355b9f9bbee3243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:24.390298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:24.392221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 854/1722] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:24.392755 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:24.404992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 869/1723] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:23 2022] DELETE /identity/v3/projects/e581913bddbe46e38355b9f9bbee3243 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:36:24.405847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd92f10-5942-4f85-b2af-18b0a69ad054 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:24.406636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:24.410386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:24.418247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2cd92f10-5942-4f85-b2af-18b0a69ad054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:24.713196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd92f10-5942-4f85-b2af-18b0a69ad054 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:24.766365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb65fe2-165b-477e-8f7a-5714f177c827 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:24.790182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:24.791423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:24.791918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:24.792396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] PATH_INFO: `/v3/users/4b42d2d116ec4591af8cc120ce2f9881` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:24.794704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b42d2d116ec4591af8cc120ce2f9881)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:24.797031 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:24.850767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daeb6d4-666b-4292-af6c-a6594f2089fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:24.853126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeb6d4-666b-4292-af6c-a6594f2089fe service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:24.854032 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeb6d4-666b-4292-af6c-a6594f2089fe service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:24.854032 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeb6d4-666b-4292-af6c-a6594f2089fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:24.869340 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0daeb6d4-666b-4292-af6c-a6594f2089fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:24.947176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b42d2d116ec4591af8cc120ce2f9881'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:24.948242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b42d2d116ec4591af8cc120ce2f9881'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:24.968760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b42d2d116ec4591af8cc120ce2f9881'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:24.998724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:25.002095 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b42d2d116ec4591af8cc120ce2f9881'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:25.004590 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b42d2d116ec4591af8cc120ce2f9881'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:25.023711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:25.054467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 870/1724] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:24 2022] DELETE /identity/v3/users/4b42d2d116ec4591af8cc120ce2f9881 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:25.058585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:25.065670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:25.104926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daeb6d4-666b-4292-af6c-a6594f2089fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:25.112652 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daeb6d4-666b-4292-af6c-a6594f2089fe service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:25.143366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 855/1725] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 738 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:25.248530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daeb6d4-666b-4292-af6c-a6594f2089fe service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:25.250552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0daeb6d4-666b-4292-af6c-a6594f2089fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:25.329639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285e027-89a5-48e1-b447-8fa3393943b6 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:25.362460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fd023-ecf7-4fb8-a456-03cfd840416c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:25.363476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fd023-ecf7-4fb8-a456-03cfd840416c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:25.363626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fd023-ecf7-4fb8-a456-03cfd840416c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:25.364418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fd023-ecf7-4fb8-a456-03cfd840416c admin admin] PATH_INFO: `/v3/users/f90fa3b5a1d5480aa1b81f06cc4a6ed6` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:25.368343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8fd023-ecf7-4fb8-a456-03cfd840416c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f90fa3b5a1d5480aa1b81f06cc4a6ed6)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:25.379111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8fd023-ecf7-4fb8-a456-03cfd840416c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:25.562423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fe8fd023-ecf7-4fb8-a456-03cfd840416c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f90fa3b5a1d5480aa1b81f06cc4a6ed6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:25.570250 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fe8fd023-ecf7-4fb8-a456-03cfd840416c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f90fa3b5a1d5480aa1b81f06cc4a6ed6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:25.628947 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fe8fd023-ecf7-4fb8-a456-03cfd840416c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f90fa3b5a1d5480aa1b81f06cc4a6ed6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:25.648720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daeb6d4-666b-4292-af6c-a6594f2089fe service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:25.717435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fe8fd023-ecf7-4fb8-a456-03cfd840416c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f90fa3b5a1d5480aa1b81f06cc4a6ed6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:25.733416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fe8fd023-ecf7-4fb8-a456-03cfd840416c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f90fa3b5a1d5480aa1b81f06cc4a6ed6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:25.801038 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 871/1726] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:25 2022] DELETE /identity/v3/users/f90fa3b5a1d5480aa1b81f06cc4a6ed6 => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:25.805771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e2e2-3488-4aef-b11f-f032d400cf7b None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:25.806072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e2e2-3488-4aef-b11f-f032d400cf7b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:25.806358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e2e2-3488-4aef-b11f-f032d400cf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:25.875885 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8b8d9-ead7-4646-821a-21f87072923c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:25.885030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8b8d9-ead7-4646-821a-21f87072923c service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:25.886971 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8b8d9-ead7-4646-821a-21f87072923c service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:25.887921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8b8d9-ead7-4646-821a-21f87072923c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:25.894394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-55a8b8d9-ead7-4646-821a-21f87072923c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:25.944543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8b8d9-ead7-4646-821a-21f87072923c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:25.956422 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8b8d9-ead7-4646-821a-21f87072923c service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:25.960530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-9635e2e2-3488-4aef-b11f-f032d400cf7b None None] MFA Rules not processed for user `3e6fe6fa3fc14cca97064b49403eb68b`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:25.968765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9635e2e2-3488-4aef-b11f-f032d400cf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:26.007925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 856/1727] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 759 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:36:26.263382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 872/1728] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:26.286732 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecb9ec-83d6-40c7-aaa7-d8ccc511ceb8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:26.287449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecb9ec-83d6-40c7-aaa7-d8ccc511ceb8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:26.287967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecb9ec-83d6-40c7-aaa7-d8ccc511ceb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:26.367354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-68ecb9ec-83d6-40c7-aaa7-d8ccc511ceb8 None None] MFA Rules not processed for user `3e6fe6fa3fc14cca97064b49403eb68b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:26.369376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-68ecb9ec-83d6-40c7-aaa7-d8ccc511ceb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:26.411858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e2e2-3488-4aef-b11f-f032d400cf7b None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:26.416081 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 857/1729] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:26.420012 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9635e2e2-3488-4aef-b11f-f032d400cf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:26.420186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecb9ec-83d6-40c7-aaa7-d8ccc511ceb8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:26.424397 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-68ecb9ec-83d6-40c7-aaa7-d8ccc511ceb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:26.443982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:26.657253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:26.680571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e2e2-3488-4aef-b11f-f032d400cf7b None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:26.701339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693f8e1-ca87-4669-9546-988b5c3455c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:26.702241 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f8e1-ca87-4669-9546-988b5c3455c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:26.702525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f8e1-ca87-4669-9546-988b5c3455c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:26.702927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f8e1-ca87-4669-9546-988b5c3455c9 admin admin] PATH_INFO: `/v3/projects/ec2b708c4e2d45aaaa213af9c05c359f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:26.705846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693f8e1-ca87-4669-9546-988b5c3455c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2b708c4e2d45aaaa213af9c05c359f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:26.708416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693f8e1-ca87-4669-9546-988b5c3455c9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:26.710440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecb9ec-83d6-40c7-aaa7-d8ccc511ceb8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:26.729470 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a185d1e-1dd6-4451-a5b9-0ad35d5ac74f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:26.730729 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a185d1e-1dd6-4451-a5b9-0ad35d5ac74f service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:26.734685 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a185d1e-1dd6-4451-a5b9-0ad35d5ac74f service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:26.735169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a185d1e-1dd6-4451-a5b9-0ad35d5ac74f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:26.754465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a185d1e-1dd6-4451-a5b9-0ad35d5ac74f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:26.760019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a185d1e-1dd6-4451-a5b9-0ad35d5ac74f service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:26.787733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 858/1730] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:26.790508 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a185d1e-1dd6-4451-a5b9-0ad35d5ac74f service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:26.797140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2a185d1e-1dd6-4451-a5b9-0ad35d5ac74f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:26.840653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a185d1e-1dd6-4451-a5b9-0ad35d5ac74f service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:26.849000 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dc4eb-38f7-49b2-a26f-e7771135550f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:26.850837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dc4eb-38f7-49b2-a26f-e7771135550f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:26.857144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dc4eb-38f7-49b2-a26f-e7771135550f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:26.857749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dc4eb-38f7-49b2-a26f-e7771135550f admin admin] PATH_INFO: `/v3/projects/2eafbcdcfa404458ac0a0472ef54c27c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:26.860884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812dc4eb-38f7-49b2-a26f-e7771135550f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eafbcdcfa404458ac0a0472ef54c27c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:26.864928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812dc4eb-38f7-49b2-a26f-e7771135550f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:26.988447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-812dc4eb-38f7-49b2-a26f-e7771135550f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eafbcdcfa404458ac0a0472ef54c27c', 'request_id': 'req-812dc4eb-38f7-49b2-a26f-e7771135550f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.000581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-812dc4eb-38f7-49b2-a26f-e7771135550f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eafbcdcfa404458ac0a0472ef54c27c', 'request_id': 'req-812dc4eb-38f7-49b2-a26f-e7771135550f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.020158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-812dc4eb-38f7-49b2-a26f-e7771135550f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eafbcdcfa404458ac0a0472ef54c27c', 'request_id': 'req-812dc4eb-38f7-49b2-a26f-e7771135550f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.034158 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d693f8e1-ca87-4669-9546-988b5c3455c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2b708c4e2d45aaaa213af9c05c359f', 'request_id': 'req-d693f8e1-ca87-4669-9546-988b5c3455c9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.050430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-812dc4eb-38f7-49b2-a26f-e7771135550f admin admin] The token cache is being invalidate because project 2eafbcdcfa404458ac0a0472ef54c27c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:27.050809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-812dc4eb-38f7-49b2-a26f-e7771135550f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.051594 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-812dc4eb-38f7-49b2-a26f-e7771135550f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.062429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d693f8e1-ca87-4669-9546-988b5c3455c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2b708c4e2d45aaaa213af9c05c359f', 'request_id': 'req-d693f8e1-ca87-4669-9546-988b5c3455c9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.074523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 859/1731] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:26 2022] DELETE /identity/v3/projects/2eafbcdcfa404458ac0a0472ef54c27c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:27.082494 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7788d5b-dab0-41ac-9c18-d792c8f174d0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:27.083349 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7788d5b-dab0-41ac-9c18-d792c8f174d0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:27.083791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7788d5b-dab0-41ac-9c18-d792c8f174d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:27.091714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d693f8e1-ca87-4669-9546-988b5c3455c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2b708c4e2d45aaaa213af9c05c359f', 'request_id': 'req-d693f8e1-ca87-4669-9546-988b5c3455c9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.134918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c7788d5b-dab0-41ac-9c18-d792c8f174d0 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:27.135125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d693f8e1-ca87-4669-9546-988b5c3455c9 admin admin] The token cache is being invalidate because project ec2b708c4e2d45aaaa213af9c05c359f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:27.135411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d693f8e1-ca87-4669-9546-988b5c3455c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.141060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d693f8e1-ca87-4669-9546-988b5c3455c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.141342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c7788d5b-dab0-41ac-9c18-d792c8f174d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:27.143398 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 873/1732] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:26 2022] DELETE /identity/v3/projects/ec2b708c4e2d45aaaa213af9c05c359f => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:27.156379 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1eb862-260c-4ab0-909e-b6931cba104d None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:27.156825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1eb862-260c-4ab0-909e-b6931cba104d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:27.157382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1eb862-260c-4ab0-909e-b6931cba104d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:27.214348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-0b1eb862-260c-4ab0-909e-b6931cba104d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:27.217123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0b1eb862-260c-4ab0-909e-b6931cba104d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:27.329703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 860/1733] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:27.339613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782f62f-1972-47c9-accc-fceef02dadc2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:27.340073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782f62f-1972-47c9-accc-fceef02dadc2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:27.340526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782f62f-1972-47c9-accc-fceef02dadc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:27.401760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b782f62f-1972-47c9-accc-fceef02dadc2 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:27.402983 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b782f62f-1972-47c9-accc-fceef02dadc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:27.441015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 861/1734] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:27.442127 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782f62f-1972-47c9-accc-fceef02dadc2 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:27.443829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b782f62f-1972-47c9-accc-fceef02dadc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:27.462970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 874/1735] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:27.465362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1eb862-260c-4ab0-909e-b6931cba104d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:27.481318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782f62f-1972-47c9-accc-fceef02dadc2 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:27.500706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:27.502113 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:27.502837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:27.503376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] PATH_INFO: `/v3/projects/c553a9f4679e4d6f9f7e23cb5d1f652f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:27.506232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c553a9f4679e4d6f9f7e23cb5d1f652f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:27.508929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:27.510801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1eb862-260c-4ab0-909e-b6931cba104d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:27.537672 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67da1ed-34d2-4206-bb53-439c9937acd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:27.538623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67da1ed-34d2-4206-bb53-439c9937acd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:27.539087 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67da1ed-34d2-4206-bb53-439c9937acd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:27.539373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67da1ed-34d2-4206-bb53-439c9937acd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:27.540115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67da1ed-34d2-4206-bb53-439c9937acd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:27.540421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67da1ed-34d2-4206-bb53-439c9937acd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:27.542067 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67da1ed-34d2-4206-bb53-439c9937acd1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:27.580338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 875/1736] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:27.581902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b014-c813-40d1-a44b-8b700ed36327 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:27.582194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b014-c813-40d1-a44b-8b700ed36327 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:27.582475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b014-c813-40d1-a44b-8b700ed36327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:27.627609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-8802b014-c813-40d1-a44b-8b700ed36327 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:27.628849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8802b014-c813-40d1-a44b-8b700ed36327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:27.661668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c553a9f4679e4d6f9f7e23cb5d1f652f', 'request_id': 'req-aeacaea6-37e0-4b96-87db-47c83f4da460'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.674743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:27.677335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c553a9f4679e4d6f9f7e23cb5d1f652f', 'request_id': 'req-aeacaea6-37e0-4b96-87db-47c83f4da460'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.696313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c553a9f4679e4d6f9f7e23cb5d1f652f', 'request_id': 'req-aeacaea6-37e0-4b96-87db-47c83f4da460'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.723139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:27.727569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] The token cache is being invalidate because project c553a9f4679e4d6f9f7e23cb5d1f652f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:27.728875 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.730486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:27.737562 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 862/1737] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:27 2022] DELETE /identity/v3/projects/c553a9f4679e4d6f9f7e23cb5d1f652f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:27.739878 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:27.749332 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:27.919249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 876/1738] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:27.921325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802b014-c813-40d1-a44b-8b700ed36327 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:27.926049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8802b014-c813-40d1-a44b-8b700ed36327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:27.961858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802b014-c813-40d1-a44b-8b700ed36327 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:27.981871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:27.984305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:27.986149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:27.986585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] PATH_INFO: `/v3/users/49856c7cf5644e8c80920595460039aa` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:27.989481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49856c7cf5644e8c80920595460039aa)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:27.993771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:28.050878 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacaea6-37e0-4b96-87db-47c83f4da460 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:28.075367 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daccb82-c73d-4a2f-88a9-9a2824520a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:28.077484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daccb82-c73d-4a2f-88a9-9a2824520a1d admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:28.080190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daccb82-c73d-4a2f-88a9-9a2824520a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:28.084394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daccb82-c73d-4a2f-88a9-9a2824520a1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:28.085786 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daccb82-c73d-4a2f-88a9-9a2824520a1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:28.095571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daccb82-c73d-4a2f-88a9-9a2824520a1d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:28.140797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49856c7cf5644e8c80920595460039aa'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:28.142625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49856c7cf5644e8c80920595460039aa'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:28.159642 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49856c7cf5644e8c80920595460039aa'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:28.211624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49856c7cf5644e8c80920595460039aa'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:28.214855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 863/1739] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:27 2022] POST /identity/v3/projects => generated 375 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:28.216255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daccb82-c73d-4a2f-88a9-9a2824520a1d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:28.220517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49856c7cf5644e8c80920595460039aa'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:28.226376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7daccb82-c73d-4a2f-88a9-9a2824520a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:28.313354 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 877/1740] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:27 2022] DELETE /identity/v3/users/49856c7cf5644e8c80920595460039aa => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:28.321858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:28.330643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:28.506924 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daccb82-c73d-4a2f-88a9-9a2824520a1d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:28.560734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da2639-8a31-4d74-98d7-c67ca43042b7 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:28.575120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b958cd0-c0f3-4c7e-86ba-e5db0a5b2a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:28.576610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b958cd0-c0f3-4c7e-86ba-e5db0a5b2a72 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:28.577029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b958cd0-c0f3-4c7e-86ba-e5db0a5b2a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:28.577424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b958cd0-c0f3-4c7e-86ba-e5db0a5b2a72 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:28.578332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b958cd0-c0f3-4c7e-86ba-e5db0a5b2a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:28.580906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b958cd0-c0f3-4c7e-86ba-e5db0a5b2a72 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:28.629685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 878/1741] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:28 2022] POST /identity/v3/users => generated 288 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:28.631192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b958cd0-c0f3-4c7e-86ba-e5db0a5b2a72 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:28.652840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b958cd0-c0f3-4c7e-86ba-e5db0a5b2a72 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:28.664430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ed463-6826-40f8-9670-6e477c86f603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:28.665733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ed463-6826-40f8-9670-6e477c86f603 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:28.666122 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ed463-6826-40f8-9670-6e477c86f603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:28.666411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ed463-6826-40f8-9670-6e477c86f603 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:28.667167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ed463-6826-40f8-9670-6e477c86f603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:28.667480 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ed463-6826-40f8-9670-6e477c86f603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:28.670765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a0da0-fc55-490c-a64d-4090d90de66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:28.671449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a0da0-fc55-490c-a64d-4090d90de66e admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:28.671733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a0da0-fc55-490c-a64d-4090d90de66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:28.672015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a0da0-fc55-490c-a64d-4090d90de66e admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:28.672801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a0da0-fc55-490c-a64d-4090d90de66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:28.674633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a0da0-fc55-490c-a64d-4090d90de66e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:28.675961 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ed463-6826-40f8-9670-6e477c86f603 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:28.703328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 879/1742] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:28.704226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a0da0-fc55-490c-a64d-4090d90de66e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:28.707036 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 864/1743] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 491 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:28.712541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ed463-6826-40f8-9670-6e477c86f603 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:28.733711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a0da0-fc55-490c-a64d-4090d90de66e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:28.737468 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ed463-6826-40f8-9670-6e477c86f603 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:28.746053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303b4c8-62d1-43f4-8c62-86f09353787f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:28.746749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303b4c8-62d1-43f4-8c62-86f09353787f admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:28.747193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303b4c8-62d1-43f4-8c62-86f09353787f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:28.747417 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303b4c8-62d1-43f4-8c62-86f09353787f admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:28.749451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303b4c8-62d1-43f4-8c62-86f09353787f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:28.751372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303b4c8-62d1-43f4-8c62-86f09353787f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:28.753689 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147cc8c-f9af-4523-959b-f772d86b89ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:28.754471 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147cc8c-f9af-4523-959b-f772d86b89ee admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:28.755333 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147cc8c-f9af-4523-959b-f772d86b89ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:28.755333 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147cc8c-f9af-4523-959b-f772d86b89ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:28.755840 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147cc8c-f9af-4523-959b-f772d86b89ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:28.758087 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147cc8c-f9af-4523-959b-f772d86b89ee admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:28.780941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 880/1744] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:28.784916 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:28.786008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303b4c8-62d1-43f4-8c62-86f09353787f admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:28.800058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:28.806427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303b4c8-62d1-43f4-8c62-86f09353787f admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:28.822693 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 865/1745] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:28 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:36:28.824855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e61d95-64ad-4934-b79b-92e313085348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:28.826955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147cc8c-f9af-4523-959b-f772d86b89ee admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:28.828513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e61d95-64ad-4934-b79b-92e313085348 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:28.828513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e61d95-64ad-4934-b79b-92e313085348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:28.828767 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e61d95-64ad-4934-b79b-92e313085348 admin admin] PATH_INFO: `/v3/projects/122ab0b07ec1444fa7565431811ca511/users/1ede400146594b19ad3d922788ae6fe0/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:28.877386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e61d95-64ad-4934-b79b-92e313085348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=122ab0b07ec1444fa7565431811ca511, user_id=1ede400146594b19ad3d922788ae6fe0, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:28.880518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e61d95-64ad-4934-b79b-92e313085348 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:28.933977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 881/1746] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:28 2022] PUT /identity/v3/projects/122ab0b07ec1444fa7565431811ca511/users/1ede400146594b19ad3d922788ae6fe0/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:28.941307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878e0c5-991e-4ff9-a0be-01ddb0617776 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:28.944620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878e0c5-991e-4ff9-a0be-01ddb0617776 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:28.945220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878e0c5-991e-4ff9-a0be-01ddb0617776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:29.028050 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1878e0c5-991e-4ff9-a0be-01ddb0617776 None None] MFA Rules not processed for user `1ede400146594b19ad3d922788ae6fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:29.029770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1878e0c5-991e-4ff9-a0be-01ddb0617776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:29.037628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147cc8c-f9af-4523-959b-f772d86b89ee admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:29.189616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a484c5-8fc1-4df2-aec1-f9cea6e58574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:29.199708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a484c5-8fc1-4df2-aec1-f9cea6e58574 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:29.200443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 882/1747] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:29.201048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a484c5-8fc1-4df2-aec1-f9cea6e58574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:29.201722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a484c5-8fc1-4df2-aec1-f9cea6e58574 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:29.202353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878e0c5-991e-4ff9-a0be-01ddb0617776 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:29.207401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a484c5-8fc1-4df2-aec1-f9cea6e58574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:29.210617 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a484c5-8fc1-4df2-aec1-f9cea6e58574 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:29.230430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878e0c5-991e-4ff9-a0be-01ddb0617776 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:29.249142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713551d-d907-4886-8a8a-a38c76ac698d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:29.252328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713551d-d907-4886-8a8a-a38c76ac698d admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:29.253279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713551d-d907-4886-8a8a-a38c76ac698d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:29.255364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713551d-d907-4886-8a8a-a38c76ac698d admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:29.256566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713551d-d907-4886-8a8a-a38c76ac698d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:29.262529 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 866/1748] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:28 2022] POST /identity/v3/users => generated 290 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:29.263080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713551d-d907-4886-8a8a-a38c76ac698d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:29.264907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a484c5-8fc1-4df2-aec1-f9cea6e58574 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:29.312600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a484c5-8fc1-4df2-aec1-f9cea6e58574 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:29.340240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bcfd0-787e-4db7-801e-59e914574e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:29.341398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bcfd0-787e-4db7-801e-59e914574e0a admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:29.343421 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bcfd0-787e-4db7-801e-59e914574e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:29.343991 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bcfd0-787e-4db7-801e-59e914574e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:29.345805 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bcfd0-787e-4db7-801e-59e914574e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:29.350132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 883/1749] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:29 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:29.351101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713551d-d907-4886-8a8a-a38c76ac698d admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:29.352082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bcfd0-787e-4db7-801e-59e914574e0a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:29.383886 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 867/1750] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:29.388601 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bcfd0-787e-4db7-801e-59e914574e0a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:29.431506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bcfd0-787e-4db7-801e-59e914574e0a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:29.801220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713551d-d907-4886-8a8a-a38c76ac698d admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:29.851152 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a1bd-a94f-40e0-bad9-4ce12adceb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:29.858013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a1bd-a94f-40e0-bad9-4ce12adceb9d admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:29.862472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a1bd-a94f-40e0-bad9-4ce12adceb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:29.863335 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a1bd-a94f-40e0-bad9-4ce12adceb9d admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:29.868087 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0a1bd-a94f-40e0-bad9-4ce12adceb9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:29.882502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0a1bd-a94f-40e0-bad9-4ce12adceb9d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:29.905184 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe433fb2-4a3e-4394-81cb-591d997f7739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:29.905938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe433fb2-4a3e-4394-81cb-591d997f7739 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:29.906220 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe433fb2-4a3e-4394-81cb-591d997f7739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:29.906479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe433fb2-4a3e-4394-81cb-591d997f7739 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:29.907242 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe433fb2-4a3e-4394-81cb-591d997f7739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:29.914943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe433fb2-4a3e-4394-81cb-591d997f7739 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:29.944036 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 868/1751] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 555 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:29.949369 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ef70d-eb62-4d18-8e4c-64d466a35655 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:29.949669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ef70d-eb62-4d18-8e4c-64d466a35655 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:29.950076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ef70d-eb62-4d18-8e4c-64d466a35655 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:29.955576 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 884/1752] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:29 2022] POST /identity/v3/users => generated 295 bytes in 605 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:29.956505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a1bd-a94f-40e0-bad9-4ce12adceb9d admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:29.957348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 869/1753] 10.222.0.39 () {54 vars in 912 bytes} [Tue Jun 7 08:36:29 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:29.960413 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ef70d-eb62-4d18-8e4c-64d466a35655 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:29.986666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a1bd-a94f-40e0-bad9-4ce12adceb9d admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:29.995145 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ef70d-eb62-4d18-8e4c-64d466a35655 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:29.997754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5a2d2-65a0-40d1-889a-40090ee63133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:30.000258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5a2d2-65a0-40d1-889a-40090ee63133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:30.000651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5a2d2-65a0-40d1-889a-40090ee63133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:30.000807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5a2d2-65a0-40d1-889a-40090ee63133 admin admin] PATH_INFO: `/v3/projects/309fa6108864453c814cd4df478aa86b/users/8e10d9a7e727453a95c6ea88d9225f0b/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:30.008496 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a7236-225f-4aa5-87b2-e02926708f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:30.009212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a7236-225f-4aa5-87b2-e02926708f70 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:30.009490 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a7236-225f-4aa5-87b2-e02926708f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:30.009759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a7236-225f-4aa5-87b2-e02926708f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:30.010480 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a7236-225f-4aa5-87b2-e02926708f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:30.015716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a7236-225f-4aa5-87b2-e02926708f70 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:30.030495 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:30.054269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5a2d2-65a0-40d1-889a-40090ee63133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=309fa6108864453c814cd4df478aa86b, user_id=8e10d9a7e727453a95c6ea88d9225f0b, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:30.058242 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 870/1754] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:30.065541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1022be5-35ba-4c0f-8030-f35cece5d82e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:30.065541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1022be5-35ba-4c0f-8030-f35cece5d82e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:30.066021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5a2d2-65a0-40d1-889a-40090ee63133 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:30.067049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1022be5-35ba-4c0f-8030-f35cece5d82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:30.153017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 885/1755] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:29 2022] PUT /identity/v3/projects/309fa6108864453c814cd4df478aa86b/users/8e10d9a7e727453a95c6ea88d9225f0b/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:30.158879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5a2d2-65a0-40d1-889a-40090ee63133 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:30.165719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b1022be5-35ba-4c0f-8030-f35cece5d82e None None] MFA Rules not processed for user `7ddf89deb28749eda2d26ad66bc282f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:30.174350 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b1022be5-35ba-4c0f-8030-f35cece5d82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:30.234626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:30.377973 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 871/1756] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jun 7 08:36:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:30.389107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbb332-8729-48f7-a400-b7f32d399b12 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:30.389107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbb332-8729-48f7-a400-b7f32d399b12 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:30.389107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbb332-8729-48f7-a400-b7f32d399b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:30.460687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2dfbb332-8729-48f7-a400-b7f32d399b12 None None] MFA Rules not processed for user `8e10d9a7e727453a95c6ea88d9225f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:30.462330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2dfbb332-8729-48f7-a400-b7f32d399b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:30.549695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5a2d2-65a0-40d1-889a-40090ee63133 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:30.572096 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62983a3-8c42-4270-a773-9b4a35dee7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:30.573651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62983a3-8c42-4270-a773-9b4a35dee7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:30.574837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62983a3-8c42-4270-a773-9b4a35dee7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:30.574837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62983a3-8c42-4270-a773-9b4a35dee7fd admin admin] PATH_INFO: `/v3/projects/f06245f01c7e47cca85cf49888d55270/users/f9a36cab4e5c444f9bc81bf35ca81ead/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:30.641847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62983a3-8c42-4270-a773-9b4a35dee7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f06245f01c7e47cca85cf49888d55270, user_id=f9a36cab4e5c444f9bc81bf35ca81ead, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:30.654481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62983a3-8c42-4270-a773-9b4a35dee7fd admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:30.663781 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 872/1757] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:30.665690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfbb332-8729-48f7-a400-b7f32d399b12 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:30.667795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2dfbb332-8729-48f7-a400-b7f32d399b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:30.696004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfbb332-8729-48f7-a400-b7f32d399b12 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:30.706145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 886/1758] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:30 2022] PUT /identity/v3/projects/f06245f01c7e47cca85cf49888d55270/users/f9a36cab4e5c444f9bc81bf35ca81ead/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:30.707477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62983a3-8c42-4270-a773-9b4a35dee7fd admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:30.712349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e62983a3-8c42-4270-a773-9b4a35dee7fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:30.835549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:30.837246 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:30.837840 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:30.839929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] PATH_INFO: `/v3/projects/675398b7217441d4aa08ccb2ff1ddeeb` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:30.841854 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=675398b7217441d4aa08ccb2ff1ddeeb)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:30.844201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:30.923710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62983a3-8c42-4270-a773-9b4a35dee7fd admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:30.929035 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fdded-be73-4a14-be03-3f4d83d29981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:30.929906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fdded-be73-4a14-be03-3f4d83d29981 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:30.930195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fdded-be73-4a14-be03-3f4d83d29981 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:30.930476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fdded-be73-4a14-be03-3f4d83d29981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:30.951114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fdded-be73-4a14-be03-3f4d83d29981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:30.952640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fdded-be73-4a14-be03-3f4d83d29981 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:30.973163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:30.980071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 887/1759] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:30.980738 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fdded-be73-4a14-be03-3f4d83d29981 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:30.999284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '675398b7217441d4aa08ccb2ff1ddeeb', 'request_id': 'req-2280e6e7-eae4-4943-89a6-e10314b56619'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:31.021924 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '675398b7217441d4aa08ccb2ff1ddeeb', 'request_id': 'req-2280e6e7-eae4-4943-89a6-e10314b56619'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:31.026425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fdded-be73-4a14-be03-3f4d83d29981 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:31.041671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '675398b7217441d4aa08ccb2ff1ddeeb', 'request_id': 'req-2280e6e7-eae4-4943-89a6-e10314b56619'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:31.070944 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] The token cache is being invalidate because project 675398b7217441d4aa08ccb2ff1ddeeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:31.071251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:31.075614 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:31.078632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 873/1760] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:30 2022] DELETE /identity/v3/projects/675398b7217441d4aa08ccb2ff1ddeeb => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:31.079828 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:31.084167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:31.209149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc70c6-6195-400f-8602-8cbd2cc5eeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:31.210296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc70c6-6195-400f-8602-8cbd2cc5eeef admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:31.210587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc70c6-6195-400f-8602-8cbd2cc5eeef admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:31.210865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc70c6-6195-400f-8602-8cbd2cc5eeef admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:31.211590 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc70c6-6195-400f-8602-8cbd2cc5eeef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:31.213610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc70c6-6195-400f-8602-8cbd2cc5eeef admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:31.278957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 888/1761] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:30 2022] POST /identity/v3/projects => generated 381 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:31.281364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf261c3-c8ae-4560-b424-15eabc77548a None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:31.281872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf261c3-c8ae-4560-b424-15eabc77548a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:31.286748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280e6e7-eae4-4943-89a6-e10314b56619 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:31.287584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf261c3-c8ae-4560-b424-15eabc77548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:31.354011 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-9cf261c3-c8ae-4560-b424-15eabc77548a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:31.355245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9cf261c3-c8ae-4560-b424-15eabc77548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:31.414965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:31.482978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31360001-42ea-48da-ae4f-4c6a87998849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:31.483719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31360001-42ea-48da-ae4f-4c6a87998849 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:31.484029 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31360001-42ea-48da-ae4f-4c6a87998849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:31.484338 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31360001-42ea-48da-ae4f-4c6a87998849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:31.486969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31360001-42ea-48da-ae4f-4c6a87998849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:31.488737 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31360001-42ea-48da-ae4f-4c6a87998849 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:31.537306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 874/1762] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:31.539096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31360001-42ea-48da-ae4f-4c6a87998849 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:31.544417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-31360001-42ea-48da-ae4f-4c6a87998849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:31.566324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31360001-42ea-48da-ae4f-4c6a87998849 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:31.573520 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f9e1d-e6c0-4cfe-865e-2b3a328f768b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:31.574466 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f9e1d-e6c0-4cfe-865e-2b3a328f768b admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:31.575097 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f9e1d-e6c0-4cfe-865e-2b3a328f768b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:31.575417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f9e1d-e6c0-4cfe-865e-2b3a328f768b admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:31.576404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f9e1d-e6c0-4cfe-865e-2b3a328f768b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:31.578497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f9e1d-e6c0-4cfe-865e-2b3a328f768b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:31.610212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 875/1763] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:31 2022] POST /identity/v3/users => generated 297 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:31.611313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f9e1d-e6c0-4cfe-865e-2b3a328f768b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:31.633972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f9e1d-e6c0-4cfe-865e-2b3a328f768b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:31.635734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 889/1764] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:31.636858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf261c3-c8ae-4560-b424-15eabc77548a None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:31.650797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f85d20-daa8-4e06-8ecc-5aa65491c3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:31.652085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f85d20-daa8-4e06-8ecc-5aa65491c3b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:31.653336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f85d20-daa8-4e06-8ecc-5aa65491c3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:31.653738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f85d20-daa8-4e06-8ecc-5aa65491c3b3 admin admin] PATH_INFO: `/v3/projects/f06245f01c7e47cca85cf49888d55270/users/f9a36cab4e5c444f9bc81bf35ca81ead/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:31.656126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f85d20-daa8-4e06-8ecc-5aa65491c3b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f06245f01c7e47cca85cf49888d55270, user_id=f9a36cab4e5c444f9bc81bf35ca81ead, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:31.660988 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f85d20-daa8-4e06-8ecc-5aa65491c3b3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:31.680361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf261c3-c8ae-4560-b424-15eabc77548a None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:31.711971 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 876/1765] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:31 2022] PUT /identity/v3/projects/f06245f01c7e47cca85cf49888d55270/users/f9a36cab4e5c444f9bc81bf35ca81ead/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:31.713497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a4e0e-a1aa-4fdb-a195-2f011877ce67 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:31.713793 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a4e0e-a1aa-4fdb-a195-2f011877ce67 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:31.714067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a4e0e-a1aa-4fdb-a195-2f011877ce67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:31.764532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f78a4e0e-a1aa-4fdb-a195-2f011877ce67 None None] MFA Rules not processed for user `7ddf89deb28749eda2d26ad66bc282f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:31.767735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f78a4e0e-a1aa-4fdb-a195-2f011877ce67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:31.862093 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:31.884038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 877/1766] 10.222.0.41 () {58 vars in 1015 bytes} [Tue Jun 7 08:36:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:31.885710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156e0a7-ea1a-4d26-b7a6-48b0276d7694 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:31.885995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156e0a7-ea1a-4d26-b7a6-48b0276d7694 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:31.886274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156e0a7-ea1a-4d26-b7a6-48b0276d7694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:31.909647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4156e0a7-ea1a-4d26-b7a6-48b0276d7694 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:31.910889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4156e0a7-ea1a-4d26-b7a6-48b0276d7694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:31.941231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42725719-4633-4d3a-abe6-b7bd81380966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:31.942462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42725719-4633-4d3a-abe6-b7bd81380966 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:31.942985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42725719-4633-4d3a-abe6-b7bd81380966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:31.944287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42725719-4633-4d3a-abe6-b7bd81380966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:31.945348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42725719-4633-4d3a-abe6-b7bd81380966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:31.950602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42725719-4633-4d3a-abe6-b7bd81380966 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:31.990856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 890/1767] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:31.992695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b050a-90eb-40f7-bee8-6eae47756f5a None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:32.010883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b050a-90eb-40f7-bee8-6eae47756f5a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:32.011681 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b050a-90eb-40f7-bee8-6eae47756f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:32.079930 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 878/1768] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:32.080926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156e0a7-ea1a-4d26-b7a6-48b0276d7694 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:32.082421 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4156e0a7-ea1a-4d26-b7a6-48b0276d7694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:32.107200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-284b050a-90eb-40f7-bee8-6eae47756f5a None None] MFA Rules not processed for user `f9a36cab4e5c444f9bc81bf35ca81ead`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:32.108790 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-284b050a-90eb-40f7-bee8-6eae47756f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:32.122768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156e0a7-ea1a-4d26-b7a6-48b0276d7694 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:32.133438 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b26a7-e86e-4150-aa65-9a3ca6d66ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:32.135641 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b26a7-e86e-4150-aa65-9a3ca6d66ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:32.136084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b26a7-e86e-4150-aa65-9a3ca6d66ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:32.137076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b26a7-e86e-4150-aa65-9a3ca6d66ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:32.164067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b26a7-e86e-4150-aa65-9a3ca6d66ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:32.179242 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b26a7-e86e-4150-aa65-9a3ca6d66ae8 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:32.200883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 879/1769] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:32.202105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b26a7-e86e-4150-aa65-9a3ca6d66ae8 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:32.227607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b26a7-e86e-4150-aa65-9a3ca6d66ae8 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:32.245907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d65a52-87dc-467e-b93a-43b215a35b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:32.247914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d65a52-87dc-467e-b93a-43b215a35b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:32.248207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d65a52-87dc-467e-b93a-43b215a35b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:32.249116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d65a52-87dc-467e-b93a-43b215a35b6b admin admin] PATH_INFO: `/v3/projects/2b5279297481488b8b44a8872000b1b4/users/7b42a70fd6ac48b3ab6796d2b010d5e9/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:32.291275 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d65a52-87dc-467e-b93a-43b215a35b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b5279297481488b8b44a8872000b1b4, user_id=7b42a70fd6ac48b3ab6796d2b010d5e9, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:32.296280 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d65a52-87dc-467e-b93a-43b215a35b6b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:32.326700 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 880/1770] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:32 2022] PUT /identity/v3/projects/2b5279297481488b8b44a8872000b1b4/users/7b42a70fd6ac48b3ab6796d2b010d5e9/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:32.329408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d65a52-87dc-467e-b93a-43b215a35b6b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:32.487812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 891/1771] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:32.490778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70edef4-d21c-4e26-b4b1-c155a71d8255 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:32.491209 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70edef4-d21c-4e26-b4b1-c155a71d8255 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:32.491567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70edef4-d21c-4e26-b4b1-c155a71d8255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:32.522478 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d65a52-87dc-467e-b93a-43b215a35b6b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:32.537975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba458a-ceaf-4aa7-a379-2578fb0d4c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:32.538670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba458a-ceaf-4aa7-a379-2578fb0d4c7c admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:32.538953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba458a-ceaf-4aa7-a379-2578fb0d4c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:32.539620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba458a-ceaf-4aa7-a379-2578fb0d4c7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:32.539980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba458a-ceaf-4aa7-a379-2578fb0d4c7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:32.540291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba458a-ceaf-4aa7-a379-2578fb0d4c7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:32.554306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba458a-ceaf-4aa7-a379-2578fb0d4c7c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:32.608970 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 881/1772] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:32.609240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-a70edef4-d21c-4e26-b4b1-c155a71d8255 None None] MFA Rules not processed for user `aa3150d932224bb0a737a3977fff46bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:32.609865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba458a-ceaf-4aa7-a379-2578fb0d4c7c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:32.617229 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a70edef4-d21c-4e26-b4b1-c155a71d8255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:32.634319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba458a-ceaf-4aa7-a379-2578fb0d4c7c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:32.647407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f9995-92c0-47a6-aba1-78fdec1638c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:32.647686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f9995-92c0-47a6-aba1-78fdec1638c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:32.647976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f9995-92c0-47a6-aba1-78fdec1638c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:32.648460 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f9995-92c0-47a6-aba1-78fdec1638c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:32.649979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f9995-92c0-47a6-aba1-78fdec1638c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:32.653708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f9995-92c0-47a6-aba1-78fdec1638c8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:32.682178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 882/1773] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:32.683566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f9995-92c0-47a6-aba1-78fdec1638c8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:32.695098 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-076f9995-92c0-47a6-aba1-78fdec1638c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:32.766356 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:32.781739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:32.825195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f9995-92c0-47a6-aba1-78fdec1638c8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:32.833638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff57f83-2781-48a2-b5eb-3fae2782247e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:32.835486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff57f83-2781-48a2-b5eb-3fae2782247e service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:32.835775 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff57f83-2781-48a2-b5eb-3fae2782247e service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:32.836054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff57f83-2781-48a2-b5eb-3fae2782247e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:32.842188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5ff57f83-2781-48a2-b5eb-3fae2782247e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:32.897524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 892/1774] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:32.897524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70edef4-d21c-4e26-b4b1-c155a71d8255 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:32.967023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70edef4-d21c-4e26-b4b1-c155a71d8255 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:32.982506 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1366c-371e-4aae-94cb-3abdb71811b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:32.984737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1366c-371e-4aae-94cb-3abdb71811b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:32.985669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1366c-371e-4aae-94cb-3abdb71811b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:32.985968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1366c-371e-4aae-94cb-3abdb71811b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:32.990509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1366c-371e-4aae-94cb-3abdb71811b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:32.996162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1366c-371e-4aae-94cb-3abdb71811b3 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:33.042145 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff57f83-2781-48a2-b5eb-3fae2782247e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:33.043967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff57f83-2781-48a2-b5eb-3fae2782247e service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:33.107328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 883/1775] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:33.112313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff57f83-2781-48a2-b5eb-3fae2782247e service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:33.120592 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 893/1776] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:32 2022] POST /identity/v3/projects => generated 379 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:33.122733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1366c-371e-4aae-94cb-3abdb71811b3 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:33.129958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6ab1366c-371e-4aae-94cb-3abdb71811b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:33.163058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff57f83-2781-48a2-b5eb-3fae2782247e service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:33.409909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1366c-371e-4aae-94cb-3abdb71811b3 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:33.423164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39f8ff-3e8b-4c92-8b40-d0d57ecc26a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:33.424324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39f8ff-3e8b-4c92-8b40-d0d57ecc26a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:33.432678 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39f8ff-3e8b-4c92-8b40-d0d57ecc26a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:33.435899 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39f8ff-3e8b-4c92-8b40-d0d57ecc26a0 admin admin] PATH_INFO: `/v3/projects/2b5279297481488b8b44a8872000b1b4/users/7b42a70fd6ac48b3ab6796d2b010d5e9/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:33.447090 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9cb65-0d44-4d1d-b55b-70ce353bc9d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:33.447917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39f8ff-3e8b-4c92-8b40-d0d57ecc26a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b5279297481488b8b44a8872000b1b4, user_id=7b42a70fd6ac48b3ab6796d2b010d5e9, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:33.448078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9cb65-0d44-4d1d-b55b-70ce353bc9d1 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:33.448342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9cb65-0d44-4d1d-b55b-70ce353bc9d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:33.448596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9cb65-0d44-4d1d-b55b-70ce353bc9d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:33.480923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39f8ff-3e8b-4c92-8b40-d0d57ecc26a0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:33.603914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 884/1777] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:33 2022] PUT /identity/v3/projects/2b5279297481488b8b44a8872000b1b4/users/7b42a70fd6ac48b3ab6796d2b010d5e9/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:33.603914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39f8ff-3e8b-4c92-8b40-d0d57ecc26a0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:33.722147 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9cb65-0d44-4d1d-b55b-70ce353bc9d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:33.728620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9cb65-0d44-4d1d-b55b-70ce353bc9d1 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:33.758897 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 894/1778] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:33.760458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43aaef-0817-4ac6-ad02-e5916b213e3e None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:33.760956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43aaef-0817-4ac6-ad02-e5916b213e3e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:33.761479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43aaef-0817-4ac6-ad02-e5916b213e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:33.851708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1b43aaef-0817-4ac6-ad02-e5916b213e3e None None] MFA Rules not processed for user `7b42a70fd6ac48b3ab6796d2b010d5e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:33.851708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1b43aaef-0817-4ac6-ad02-e5916b213e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:34.003407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:34.169457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 895/1779] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:34.177475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b16e6-a6b9-43e3-8326-fe317a4042a8 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:34.179164 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b16e6-a6b9-43e3-8326-fe317a4042a8 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:34.179805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b16e6-a6b9-43e3-8326-fe317a4042a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:34.197958 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39f8ff-3e8b-4c92-8b40-d0d57ecc26a0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:34.245817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad09484-5c3b-4bdc-aad2-988e0f6318eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:34.246493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad09484-5c3b-4bdc-aad2-988e0f6318eb admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:34.246767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad09484-5c3b-4bdc-aad2-988e0f6318eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:34.247041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad09484-5c3b-4bdc-aad2-988e0f6318eb admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:34.247765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad09484-5c3b-4bdc-aad2-988e0f6318eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:34.257195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad09484-5c3b-4bdc-aad2-988e0f6318eb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:34.258169 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-7b1b16e6-a6b9-43e3-8326-fe317a4042a8 None None] MFA Rules not processed for user `4f603b289d964ee3ba3931948acf6788`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:34.262550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-7b1b16e6-a6b9-43e3-8326-fe317a4042a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:34.366395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 885/1780] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:33 2022] POST /identity/v3/users => generated 290 bytes in 770 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:34.367315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad09484-5c3b-4bdc-aad2-988e0f6318eb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:34.382309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8ad09484-5c3b-4bdc-aad2-988e0f6318eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:34.453528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 896/1781] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:34.465004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b16e6-a6b9-43e3-8326-fe317a4042a8 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:34.497294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:34.512565 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b16e6-a6b9-43e3-8326-fe317a4042a8 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:34.532007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e3ae3-c9cf-4c09-a3c5-5e2c46e10723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:34.537157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e3ae3-c9cf-4c09-a3c5-5e2c46e10723 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:34.540918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e3ae3-c9cf-4c09-a3c5-5e2c46e10723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:34.542418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e3ae3-c9cf-4c09-a3c5-5e2c46e10723 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:34.543386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e3ae3-c9cf-4c09-a3c5-5e2c46e10723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:34.546508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e3ae3-c9cf-4c09-a3c5-5e2c46e10723 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:34.593286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 897/1782] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:34.602450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e3ae3-c9cf-4c09-a3c5-5e2c46e10723 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:34.696747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e3ae3-c9cf-4c09-a3c5-5e2c46e10723 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:34.719335 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493d512-5720-4173-98dd-e7567a8314e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:34.724198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493d512-5720-4173-98dd-e7567a8314e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:34.728178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493d512-5720-4173-98dd-e7567a8314e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:34.728694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493d512-5720-4173-98dd-e7567a8314e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:34.729713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493d512-5720-4173-98dd-e7567a8314e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:34.734984 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493d512-5720-4173-98dd-e7567a8314e9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:34.756492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad09484-5c3b-4bdc-aad2-988e0f6318eb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:34.803975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd29be2-a936-40be-8cb1-cae820182d67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:34.804838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 898/1783] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:34.805939 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd29be2-a936-40be-8cb1-cae820182d67 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:34.807753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493d512-5720-4173-98dd-e7567a8314e9 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:34.808306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd29be2-a936-40be-8cb1-cae820182d67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:34.809270 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd29be2-a936-40be-8cb1-cae820182d67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:34.846023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd29be2-a936-40be-8cb1-cae820182d67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:34.846999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493d512-5720-4173-98dd-e7567a8314e9 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:34.850164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd29be2-a936-40be-8cb1-cae820182d67 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:34.860738 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd7985-d820-4e17-a6c4-843c39ab80cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:34.861842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd7985-d820-4e17-a6c4-843c39ab80cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:34.862323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd7985-d820-4e17-a6c4-843c39ab80cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:34.862711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd7985-d820-4e17-a6c4-843c39ab80cb admin admin] PATH_INFO: `/v3/projects/deb90f37a44d47518ed4b97f213f6bb4/users/fda71c9c7cea4b41a0a31d2ad8f20802/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:34.906980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 886/1784] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:34.946462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd7985-d820-4e17-a6c4-843c39ab80cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=deb90f37a44d47518ed4b97f213f6bb4, user_id=fda71c9c7cea4b41a0a31d2ad8f20802, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:34.949055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd7985-d820-4e17-a6c4-843c39ab80cb admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:34.998362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb717f-40b0-405f-be23-0a84983460eb None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:34.999293 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb717f-40b0-405f-be23-0a84983460eb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:34.999795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb717f-40b0-405f-be23-0a84983460eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:35.012193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 899/1785] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:34 2022] PUT /identity/v3/projects/deb90f37a44d47518ed4b97f213f6bb4/users/fda71c9c7cea4b41a0a31d2ad8f20802/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:35.026583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaaafe7-87b7-4a1b-97c6-9b28be3e8d20 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:35.027187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaaafe7-87b7-4a1b-97c6-9b28be3e8d20 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:35.027632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaaafe7-87b7-4a1b-97c6-9b28be3e8d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:35.056244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3ddb717f-40b0-405f-be23-0a84983460eb None None] MFA Rules not processed for user `3e6fe6fa3fc14cca97064b49403eb68b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:35.058360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3ddb717f-40b0-405f-be23-0a84983460eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:35.166423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-cdaaafe7-87b7-4a1b-97c6-9b28be3e8d20 None None] MFA Rules not processed for user `fda71c9c7cea4b41a0a31d2ad8f20802`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:35.167692 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-cdaaafe7-87b7-4a1b-97c6-9b28be3e8d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:35.338537 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 887/1786] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:35.556261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb717f-40b0-405f-be23-0a84983460eb None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:35.562236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 900/1787] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 536 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:35.851150 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb717f-40b0-405f-be23-0a84983460eb None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:35.866501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13958f22-e4ae-4fbb-909c-960fdf07b7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:35.870361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13958f22-e4ae-4fbb-909c-960fdf07b7bf admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:35.870722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13958f22-e4ae-4fbb-909c-960fdf07b7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:35.873366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13958f22-e4ae-4fbb-909c-960fdf07b7bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:35.878357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13958f22-e4ae-4fbb-909c-960fdf07b7bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:35.890960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13958f22-e4ae-4fbb-909c-960fdf07b7bf admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:36.004275 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 888/1788] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:35 2022] POST /identity/v3/projects => generated 383 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:36.008842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaaafe7-87b7-4a1b-97c6-9b28be3e8d20 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:36.043832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaaafe7-87b7-4a1b-97c6-9b28be3e8d20 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:36.144272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:36.170275 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13958f22-e4ae-4fbb-909c-960fdf07b7bf admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:36.334053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52e540-1736-4c73-84bd-e4169ada8ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:36.335399 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52e540-1736-4c73-84bd-e4169ada8ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:36.336074 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52e540-1736-4c73-84bd-e4169ada8ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:36.336610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52e540-1736-4c73-84bd-e4169ada8ae3 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:36.338723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52e540-1736-4c73-84bd-e4169ada8ae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:36.344253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52e540-1736-4c73-84bd-e4169ada8ae3 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:36.398483 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:36.411710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 901/1789] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:36 2022] POST /identity/v3/users => generated 298 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:36.413426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52e540-1736-4c73-84bd-e4169ada8ae3 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:36.431952 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13958f22-e4ae-4fbb-909c-960fdf07b7bf admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:36.434900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52e540-1736-4c73-84bd-e4169ada8ae3 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:36.445462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d977d2-7c32-44a2-97a8-5a61900566d6 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b42a70fd6ac48b3ab6796d2b010d5e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b5279297481488b8b44a8872000b1b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:36.446364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d977d2-7c32-44a2-97a8-5a61900566d6 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:36.446650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d977d2-7c32-44a2-97a8-5a61900566d6 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:36.447413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858ce8e-99c6-47a5-a409-9c2e07545c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:36.448083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858ce8e-99c6-47a5-a409-9c2e07545c43 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:36.448360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858ce8e-99c6-47a5-a409-9c2e07545c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:36.448631 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858ce8e-99c6-47a5-a409-9c2e07545c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:36.449077 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d977d2-7c32-44a2-97a8-5a61900566d6 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:36.449847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d977d2-7c32-44a2-97a8-5a61900566d6 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:36.450188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d977d2-7c32-44a2-97a8-5a61900566d6 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:36.451939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858ce8e-99c6-47a5-a409-9c2e07545c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:36.453793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858ce8e-99c6-47a5-a409-9c2e07545c43 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:36.455209 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d977d2-7c32-44a2-97a8-5a61900566d6 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:36.481786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 902/1790] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:36.487432 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858ce8e-99c6-47a5-a409-9c2e07545c43 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:36.507634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 889/1791] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:36:36.511627 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d977d2-7c32-44a2-97a8-5a61900566d6 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:36.528308 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858ce8e-99c6-47a5-a409-9c2e07545c43 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:36.532052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d977d2-7c32-44a2-97a8-5a61900566d6 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:36.543301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96867b97-1198-4689-9083-42cfb8351c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:36.544223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96867b97-1198-4689-9083-42cfb8351c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:36.544509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96867b97-1198-4689-9083-42cfb8351c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:36.547760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96867b97-1198-4689-9083-42cfb8351c17 admin admin] PATH_INFO: `/v3/projects/4e2ddea64c7d4063aad5e64c7c21018e/users/7d1b9131bca24b0a929fd192d42a7ef4/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:36.549455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fea509-ffd3-4843-8710-649f8a2cfd04 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b42a70fd6ac48b3ab6796d2b010d5e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b5279297481488b8b44a8872000b1b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:36.551389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fea509-ffd3-4843-8710-649f8a2cfd04 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:36.551666 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fea509-ffd3-4843-8710-649f8a2cfd04 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:36.551951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fea509-ffd3-4843-8710-649f8a2cfd04 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:36.565348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fea509-ffd3-4843-8710-649f8a2cfd04 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:36.569765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fea509-ffd3-4843-8710-649f8a2cfd04 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:36.639566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96867b97-1198-4689-9083-42cfb8351c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e2ddea64c7d4063aad5e64c7c21018e, user_id=7d1b9131bca24b0a929fd192d42a7ef4, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:36.651370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96867b97-1198-4689-9083-42cfb8351c17 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:36.712992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 903/1792] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:36 2022] PUT /identity/v3/projects/4e2ddea64c7d4063aad5e64c7c21018e/users/7d1b9131bca24b0a929fd192d42a7ef4/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:36.718436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96867b97-1198-4689-9083-42cfb8351c17 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:36.726461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 890/1793] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:36 2022] POST /identity/v3/projects => generated 354 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:36.726461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fea509-ffd3-4843-8710-649f8a2cfd04 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:36.894552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fea509-ffd3-4843-8710-649f8a2cfd04 tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:36.912010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef35fa-5a02-40dd-9a9c-d3cf9e13e14d tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b42a70fd6ac48b3ab6796d2b010d5e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b5279297481488b8b44a8872000b1b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:36.913990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef35fa-5a02-40dd-9a9c-d3cf9e13e14d tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:36.914481 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef35fa-5a02-40dd-9a9c-d3cf9e13e14d tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:36.915584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef35fa-5a02-40dd-9a9c-d3cf9e13e14d tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:36.917673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef35fa-5a02-40dd-9a9c-d3cf9e13e14d tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:36.918171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef35fa-5a02-40dd-9a9c-d3cf9e13e14d tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:36.920798 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef35fa-5a02-40dd-9a9c-d3cf9e13e14d tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:36.949038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 891/1794] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:36.995289 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96867b97-1198-4689-9083-42cfb8351c17 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:37.017774 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef35fa-5a02-40dd-9a9c-d3cf9e13e14d tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:37.032701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef35fa-5a02-40dd-9a9c-d3cf9e13e14d tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:37.048709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b42a70fd6ac48b3ab6796d2b010d5e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b5279297481488b8b44a8872000b1b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:37.051679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:37.051986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:37.052313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] PATH_INFO: `/v3/projects/70557c8487b747b7932fe3eaf56874f9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:37.058140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: Authorizing `identity:delete_project(project_id=70557c8487b747b7932fe3eaf56874f9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:37.060139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:37.198633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5818d4-9a92-4fd3-a247-8b15617cb3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:37.199324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5818d4-9a92-4fd3-a247-8b15617cb3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:37.199716 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5818d4-9a92-4fd3-a247-8b15617cb3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:37.199893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5818d4-9a92-4fd3-a247-8b15617cb3e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:37.200629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5818d4-9a92-4fd3-a247-8b15617cb3e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:37.202272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5818d4-9a92-4fd3-a247-8b15617cb3e4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:37.233133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 904/1795] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 513 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:37.233133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca496ac-b716-47e1-8ea2-19e07933e257 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:37.233133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca496ac-b716-47e1-8ea2-19e07933e257 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:37.233133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca496ac-b716-47e1-8ea2-19e07933e257 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:37.240525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 905/1796] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jun 7 08:36:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:37.241452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca496ac-b716-47e1-8ea2-19e07933e257 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:37.273586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70557c8487b747b7932fe3eaf56874f9', 'request_id': 'req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:37.288215 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70557c8487b747b7932fe3eaf56874f9', 'request_id': 'req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:37.291011 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:37.304614 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70557c8487b747b7932fe3eaf56874f9', 'request_id': 'req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:37.339929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] The token cache is being invalidate because project 70557c8487b747b7932fe3eaf56874f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:37.340706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:37.341760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a6d9dc2f-9b7b-46b8-a635-cdc3f83552fc tempest-VolumeQuotaClassesTest-1915013767 tempest-VolumeQuotaClassesTest-1915013767-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:37.346861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 892/1797] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:37 2022] DELETE /identity/v3/projects/70557c8487b747b7932fe3eaf56874f9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:37.348881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fe8f0-f7e8-4a2c-8684-ba2cd56d55bf None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:37.349641 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fe8f0-f7e8-4a2c-8684-ba2cd56d55bf None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:37.349976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fe8f0-f7e8-4a2c-8684-ba2cd56d55bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:37.398076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca496ac-b716-47e1-8ea2-19e07933e257 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:37.409336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2a5fe8f0-f7e8-4a2c-8684-ba2cd56d55bf None None] MFA Rules not processed for user `3f2a2bf913d1412f996acf4401ba7673`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:37.411613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68563e0-82b2-4773-b4ee-1762cb9a60a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:37.413922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68563e0-82b2-4773-b4ee-1762cb9a60a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:37.414169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2a5fe8f0-f7e8-4a2c-8684-ba2cd56d55bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:37.416284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68563e0-82b2-4773-b4ee-1762cb9a60a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:37.417301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68563e0-82b2-4773-b4ee-1762cb9a60a4 admin admin] PATH_INFO: `/v3/projects/4e2ddea64c7d4063aad5e64c7c21018e/users/7d1b9131bca24b0a929fd192d42a7ef4/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:37.422900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68563e0-82b2-4773-b4ee-1762cb9a60a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e2ddea64c7d4063aad5e64c7c21018e, user_id=7d1b9131bca24b0a929fd192d42a7ef4, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:37.447354 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68563e0-82b2-4773-b4ee-1762cb9a60a4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:37.507657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 906/1798] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:37 2022] PUT /identity/v3/projects/4e2ddea64c7d4063aad5e64c7c21018e/users/7d1b9131bca24b0a929fd192d42a7ef4/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:37.517587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55da84-635f-45d5-b3ce-50d6c962b6fa None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:37.520842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55da84-635f-45d5-b3ce-50d6c962b6fa None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:37.521633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55da84-635f-45d5-b3ce-50d6c962b6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:37.566125 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:37.647001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 893/1799] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jun 7 08:36:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:37.650906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-9f55da84-635f-45d5-b3ce-50d6c962b6fa None None] MFA Rules not processed for user `7d1b9131bca24b0a929fd192d42a7ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:37.654807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9f55da84-635f-45d5-b3ce-50d6c962b6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:37.661327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fe8f0-f7e8-4a2c-8684-ba2cd56d55bf None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:37.663361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2a5fe8f0-f7e8-4a2c-8684-ba2cd56d55bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:37.689292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fe8f0-f7e8-4a2c-8684-ba2cd56d55bf None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:37.826978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9483a9-9c86-4b2c-a951-dea0e7166467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:37.832135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9483a9-9c86-4b2c-a951-dea0e7166467 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:37.832461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9483a9-9c86-4b2c-a951-dea0e7166467 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:37.832998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9483a9-9c86-4b2c-a951-dea0e7166467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:37.872796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9483a9-9c86-4b2c-a951-dea0e7166467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:37.882334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9483a9-9c86-4b2c-a951-dea0e7166467 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:37.908804 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 894/1800] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:37.934727 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 907/1801] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:37.942190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9483a9-9c86-4b2c-a951-dea0e7166467 service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:37.944738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ae9483a9-9c86-4b2c-a951-dea0e7166467 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:37.949370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55da84-635f-45d5-b3ce-50d6c962b6fa None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:37.950881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9f55da84-635f-45d5-b3ce-50d6c962b6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:38.019959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55da84-635f-45d5-b3ce-50d6c962b6fa None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:38.024445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabdd37-28ce-44cf-8a5b-accfcb7f775a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:38.025296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabdd37-28ce-44cf-8a5b-accfcb7f775a service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:38.025658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabdd37-28ce-44cf-8a5b-accfcb7f775a service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:38.026008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabdd37-28ce-44cf-8a5b-accfcb7f775a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:38.038235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dabdd37-28ce-44cf-8a5b-accfcb7f775a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:38.040058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dabdd37-28ce-44cf-8a5b-accfcb7f775a service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:38.052305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 908/1802] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:38.120252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9483a9-9c86-4b2c-a951-dea0e7166467 service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:38.135669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e45292-bfe8-4f0b-8ec3-8a2231cfc020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:38.136521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e45292-bfe8-4f0b-8ec3-8a2231cfc020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:38.136978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e45292-bfe8-4f0b-8ec3-8a2231cfc020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:38.137109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e45292-bfe8-4f0b-8ec3-8a2231cfc020 admin admin] PATH_INFO: `/v3/users/8e10d9a7e727453a95c6ea88d9225f0b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:38.141698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e45292-bfe8-4f0b-8ec3-8a2231cfc020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e10d9a7e727453a95c6ea88d9225f0b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:38.143028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e45292-bfe8-4f0b-8ec3-8a2231cfc020 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:38.210771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabdd37-28ce-44cf-8a5b-accfcb7f775a service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:38.222953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabdd37-28ce-44cf-8a5b-accfcb7f775a service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:38.253820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1418a3-569f-4c0e-85d6-3a1ad7781000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:38.254515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1418a3-569f-4c0e-85d6-3a1ad7781000 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:38.254803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1418a3-569f-4c0e-85d6-3a1ad7781000 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:38.255083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1418a3-569f-4c0e-85d6-3a1ad7781000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:38.267324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-3a1418a3-569f-4c0e-85d6-3a1ad7781000 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:38.310167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e3e45292-bfe8-4f0b-8ec3-8a2231cfc020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e10d9a7e727453a95c6ea88d9225f0b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:38.311665 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e3e45292-bfe8-4f0b-8ec3-8a2231cfc020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e10d9a7e727453a95c6ea88d9225f0b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:38.332208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e3e45292-bfe8-4f0b-8ec3-8a2231cfc020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e10d9a7e727453a95c6ea88d9225f0b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:38.333124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e3e45292-bfe8-4f0b-8ec3-8a2231cfc020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e10d9a7e727453a95c6ea88d9225f0b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:38.336854 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:38.366407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e3e45292-bfe8-4f0b-8ec3-8a2231cfc020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e10d9a7e727453a95c6ea88d9225f0b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:38.408040 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 895/1803] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:37 2022] DELETE /identity/v3/users/8e10d9a7e727453a95c6ea88d9225f0b => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:38.416066 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73a97d-d8d6-4295-b20a-2d7ce2277d44 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:38.416357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73a97d-d8d6-4295-b20a-2d7ce2277d44 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:38.416806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73a97d-d8d6-4295-b20a-2d7ce2277d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:38.420510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:38.468736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7c73a97d-d8d6-4295-b20a-2d7ce2277d44 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:38.470590 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7c73a97d-d8d6-4295-b20a-2d7ce2277d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:38.478848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1418a3-569f-4c0e-85d6-3a1ad7781000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:38.480787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1418a3-569f-4c0e-85d6-3a1ad7781000 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:38.497523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 909/1804] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:38.569162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 896/1805] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:38.578462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1418a3-569f-4c0e-85d6-3a1ad7781000 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:38.580341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-3a1418a3-569f-4c0e-85d6-3a1ad7781000 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:38.668743 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1418a3-569f-4c0e-85d6-3a1ad7781000 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:38.810172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eba163-c3b8-4b5e-af1c-ca69b2d8822b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:38.812956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eba163-c3b8-4b5e-af1c-ca69b2d8822b service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:38.814611 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eba163-c3b8-4b5e-af1c-ca69b2d8822b service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:38.815707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eba163-c3b8-4b5e-af1c-ca69b2d8822b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:38.867792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eba163-c3b8-4b5e-af1c-ca69b2d8822b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:38.874748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eba163-c3b8-4b5e-af1c-ca69b2d8822b service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:38.920092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 910/1806] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:36:39.614156 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73a97d-d8d6-4295-b20a-2d7ce2277d44 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:39.618108 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7c73a97d-d8d6-4295-b20a-2d7ce2277d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:39.657517 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73a97d-d8d6-4295-b20a-2d7ce2277d44 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:39.681538 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:39.685729 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:39.686004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:39.687346 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] PATH_INFO: `/v3/projects/309fa6108864453c814cd4df478aa86b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:39.690775 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=309fa6108864453c814cd4df478aa86b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:39.713810 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:39.814073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '309fa6108864453c814cd4df478aa86b', 'request_id': 'req-1786a2d0-4db2-4154-a375-9de9fb36d444'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:39.828133 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '309fa6108864453c814cd4df478aa86b', 'request_id': 'req-1786a2d0-4db2-4154-a375-9de9fb36d444'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:39.851254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '309fa6108864453c814cd4df478aa86b', 'request_id': 'req-1786a2d0-4db2-4154-a375-9de9fb36d444'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:39.911467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] The token cache is being invalidate because project 309fa6108864453c814cd4df478aa86b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:39.912833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:39.917130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:39.922820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 897/1807] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:39 2022] DELETE /identity/v3/projects/309fa6108864453c814cd4df478aa86b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:39.928779 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eba163-c3b8-4b5e-af1c-ca69b2d8822b service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:39.940174 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eba163-c3b8-4b5e-af1c-ca69b2d8822b service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:40.047330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64964b15-592b-4aaa-a20f-b265be1d2489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:40.049008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64964b15-592b-4aaa-a20f-b265be1d2489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:40.049454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64964b15-592b-4aaa-a20f-b265be1d2489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:40.049855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64964b15-592b-4aaa-a20f-b265be1d2489 admin admin] PATH_INFO: `/v3/users/7b42a70fd6ac48b3ab6796d2b010d5e9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:40.055164 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64964b15-592b-4aaa-a20f-b265be1d2489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b42a70fd6ac48b3ab6796d2b010d5e9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:40.058839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64964b15-592b-4aaa-a20f-b265be1d2489 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:40.199402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:40.205888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-64964b15-592b-4aaa-a20f-b265be1d2489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b42a70fd6ac48b3ab6796d2b010d5e9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:40.209455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-64964b15-592b-4aaa-a20f-b265be1d2489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b42a70fd6ac48b3ab6796d2b010d5e9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:40.266230 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-64964b15-592b-4aaa-a20f-b265be1d2489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b42a70fd6ac48b3ab6796d2b010d5e9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:40.326741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-64964b15-592b-4aaa-a20f-b265be1d2489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b42a70fd6ac48b3ab6796d2b010d5e9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:40.338055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-64964b15-592b-4aaa-a20f-b265be1d2489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b42a70fd6ac48b3ab6796d2b010d5e9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:40.362586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 911/1808] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:39 2022] DELETE /identity/v3/users/7b42a70fd6ac48b3ab6796d2b010d5e9 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:40.496579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:40.498108 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:40.594212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786a2d0-4db2-4154-a375-9de9fb36d444 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:40.613676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:40.757721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94eabe7-bad8-444f-81a7-e81a540c5dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:40.758315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94eabe7-bad8-444f-81a7-e81a540c5dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:40.758476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94eabe7-bad8-444f-81a7-e81a540c5dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:40.758753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94eabe7-bad8-444f-81a7-e81a540c5dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:40.762972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c94eabe7-bad8-444f-81a7-e81a540c5dc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:40.953535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94eabe7-bad8-444f-81a7-e81a540c5dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:40.955214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94eabe7-bad8-444f-81a7-e81a540c5dc3 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:41.003209 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 898/1809] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:41.395559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc1995-23c9-4b13-9c28-552a923925a0 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:41.399519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc1995-23c9-4b13-9c28-552a923925a0 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:41.403550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc1995-23c9-4b13-9c28-552a923925a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:41.484112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-76cc1995-23c9-4b13-9c28-552a923925a0 None None] MFA Rules not processed for user `aa3150d932224bb0a737a3977fff46bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:41.489537 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-76cc1995-23c9-4b13-9c28-552a923925a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:41.592852 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 912/1810] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:42.657392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94eabe7-bad8-444f-81a7-e81a540c5dc3 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:42.659997 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c94eabe7-bad8-444f-81a7-e81a540c5dc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:42.735330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc1995-23c9-4b13-9c28-552a923925a0 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:42.737072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-76cc1995-23c9-4b13-9c28-552a923925a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:42.788198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94eabe7-bad8-444f-81a7-e81a540c5dc3 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:42.798245 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6322bd8-9bcd-4c0c-992c-7ed53af3d5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:42.799227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6322bd8-9bcd-4c0c-992c-7ed53af3d5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:42.799640 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6322bd8-9bcd-4c0c-992c-7ed53af3d5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:42.799928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6322bd8-9bcd-4c0c-992c-7ed53af3d5d7 admin admin] PATH_INFO: `/v3/users/fda71c9c7cea4b41a0a31d2ad8f20802` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:42.802726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6322bd8-9bcd-4c0c-992c-7ed53af3d5d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fda71c9c7cea4b41a0a31d2ad8f20802)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:42.806471 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6322bd8-9bcd-4c0c-992c-7ed53af3d5d7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:42.838771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc1995-23c9-4b13-9c28-552a923925a0 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:42.844827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:42.846341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:42.846635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:42.846904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a admin admin] PATH_INFO: `/v3/projects/2b5279297481488b8b44a8872000b1b4` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:42.848181 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b5279297481488b8b44a8872000b1b4)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:42.850197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:42.908443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c6322bd8-9bcd-4c0c-992c-7ed53af3d5d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fda71c9c7cea4b41a0a31d2ad8f20802'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:42.909797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c6322bd8-9bcd-4c0c-992c-7ed53af3d5d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fda71c9c7cea4b41a0a31d2ad8f20802'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:42.924393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c6322bd8-9bcd-4c0c-992c-7ed53af3d5d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fda71c9c7cea4b41a0a31d2ad8f20802'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:42.925033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c6322bd8-9bcd-4c0c-992c-7ed53af3d5d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fda71c9c7cea4b41a0a31d2ad8f20802'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:42.938615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b5279297481488b8b44a8872000b1b4', 'request_id': 'req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:42.941142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:42.952401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b5279297481488b8b44a8872000b1b4', 'request_id': 'req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:42.953201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c6322bd8-9bcd-4c0c-992c-7ed53af3d5d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fda71c9c7cea4b41a0a31d2ad8f20802'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:42.971013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b5279297481488b8b44a8872000b1b4', 'request_id': 'req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:42.986016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:43.004518 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 899/1811] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:42 2022] DELETE /identity/v3/users/fda71c9c7cea4b41a0a31d2ad8f20802 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:43.004744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a admin admin] The token cache is being invalidate because project 2b5279297481488b8b44a8872000b1b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:43.004744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:43.007999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dba157-6a03-4cb0-a6d5-2e09b6d97b1c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:43.008539 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dba157-6a03-4cb0-a6d5-2e09b6d97b1c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:43.008659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dba157-6a03-4cb0-a6d5-2e09b6d97b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:43.015528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-09b8d3fd-0253-4cb1-babd-7797aa44aa3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:43.020647 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 913/1812] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:42 2022] DELETE /identity/v3/projects/2b5279297481488b8b44a8872000b1b4 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:36:43.030580 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dc41b-f182-4d54-a39d-6382dd399b39 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:43.031118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dc41b-f182-4d54-a39d-6382dd399b39 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:43.031580 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dc41b-f182-4d54-a39d-6382dd399b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:43.045374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-07dba157-6a03-4cb0-a6d5-2e09b6d97b1c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:43.047695 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-07dba157-6a03-4cb0-a6d5-2e09b6d97b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:43.057318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-5b3dc41b-f182-4d54-a39d-6382dd399b39 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:43.058927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5b3dc41b-f182-4d54-a39d-6382dd399b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:43.173071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 914/1813] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:43.180726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a94854-9dbc-42c4-90a5-5fc8db812aa6 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:43.181128 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a94854-9dbc-42c4-90a5-5fc8db812aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:43.181327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a94854-9dbc-42c4-90a5-5fc8db812aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:43.223523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-a0a94854-9dbc-42c4-90a5-5fc8db812aa6 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:43.226318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a0a94854-9dbc-42c4-90a5-5fc8db812aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:43.245296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 915/1814] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:43.256221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a94854-9dbc-42c4-90a5-5fc8db812aa6 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:43.277409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a94854-9dbc-42c4-90a5-5fc8db812aa6 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:43.315641 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee9134-f067-4b05-a15b-d6383591427d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:43.319950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee9134-f067-4b05-a15b-d6383591427d admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:43.321712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee9134-f067-4b05-a15b-d6383591427d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:43.322427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee9134-f067-4b05-a15b-d6383591427d admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:43.323932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee9134-f067-4b05-a15b-d6383591427d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:43.324849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee9134-f067-4b05-a15b-d6383591427d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:43.331282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee9134-f067-4b05-a15b-d6383591427d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:43.369223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 916/1815] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:43.373968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee9134-f067-4b05-a15b-d6383591427d admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:43.403552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee9134-f067-4b05-a15b-d6383591427d admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:43.414741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 900/1816] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:43.422994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ba826-3562-42df-ab01-2e88f55414dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:43.423931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ba826-3562-42df-ab01-2e88f55414dc admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:43.424434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ba826-3562-42df-ab01-2e88f55414dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:43.424930 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ba826-3562-42df-ab01-2e88f55414dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:43.427896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ba826-3562-42df-ab01-2e88f55414dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:43.431061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ba826-3562-42df-ab01-2e88f55414dc admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:43.458954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dba157-6a03-4cb0-a6d5-2e09b6d97b1c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:43.460770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-07dba157-6a03-4cb0-a6d5-2e09b6d97b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:43.487890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 917/1817] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:43 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:36:43.492009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ba826-3562-42df-ab01-2e88f55414dc admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:43.564954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dba157-6a03-4cb0-a6d5-2e09b6d97b1c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:43.695509 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa5439-fdfc-495c-a775-4a957a774972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:43.696471 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa5439-fdfc-495c-a775-4a957a774972 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:43.696471 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa5439-fdfc-495c-a775-4a957a774972 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:43.696710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa5439-fdfc-495c-a775-4a957a774972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:43.701967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ba826-3562-42df-ab01-2e88f55414dc admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:43.711478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b0d83-4203-4b57-9783-ee7d82dd8782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:43.712185 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b0d83-4203-4b57-9783-ee7d82dd8782 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:43.712749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b0d83-4203-4b57-9783-ee7d82dd8782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:43.713197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b0d83-4203-4b57-9783-ee7d82dd8782 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:43.715589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b0d83-4203-4b57-9783-ee7d82dd8782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:43.719226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b0d83-4203-4b57-9783-ee7d82dd8782 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:43.747604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa5439-fdfc-495c-a775-4a957a774972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:43.751472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa5439-fdfc-495c-a775-4a957a774972 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:43.764898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 918/1818] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:43 2022] POST /identity/v3/users => generated 287 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:43.768683 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b0d83-4203-4b57-9783-ee7d82dd8782 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:43.775770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 901/1819] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:36:43.783434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b0d83-4203-4b57-9783-ee7d82dd8782 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:43.789665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a92948-c4e7-470f-a8c4-9ea2667756b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:43.790502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a92948-c4e7-470f-a8c4-9ea2667756b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:43.790839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a92948-c4e7-470f-a8c4-9ea2667756b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:43.791165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a92948-c4e7-470f-a8c4-9ea2667756b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:43.792054 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a92948-c4e7-470f-a8c4-9ea2667756b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:43.794219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a92948-c4e7-470f-a8c4-9ea2667756b9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:43.817609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 919/1820] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:36:43.820524 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa5439-fdfc-495c-a775-4a957a774972 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:43.836363 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa5439-fdfc-495c-a775-4a957a774972 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:43.865478 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2860b30-dafc-4309-87a0-811bcb6d5840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:43.866215 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2860b30-dafc-4309-87a0-811bcb6d5840 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:43.866890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2860b30-dafc-4309-87a0-811bcb6d5840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:43.866890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2860b30-dafc-4309-87a0-811bcb6d5840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:43.870222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2860b30-dafc-4309-87a0-811bcb6d5840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:43.878121 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2860b30-dafc-4309-87a0-811bcb6d5840 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:43.903205 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 902/1821] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:43.909635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a92948-c4e7-470f-a8c4-9ea2667756b9 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:43.928797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a92948-c4e7-470f-a8c4-9ea2667756b9 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:43.946126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5427f-e970-454c-bfde-44aa4a6bb89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:43.947317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5427f-e970-454c-bfde-44aa4a6bb89f admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:43.947494 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5427f-e970-454c-bfde-44aa4a6bb89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:43.947769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5427f-e970-454c-bfde-44aa4a6bb89f admin admin] PATH_INFO: `/v3/projects/c16443dacecc446194bd04317c2779f5/users/f76215c187a24cc3813d041f836da1e9/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:43.973519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5427f-e970-454c-bfde-44aa4a6bb89f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c16443dacecc446194bd04317c2779f5, user_id=f76215c187a24cc3813d041f836da1e9, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:43.987168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5427f-e970-454c-bfde-44aa4a6bb89f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:44.046413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 920/1822] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:43 2022] PUT /identity/v3/projects/c16443dacecc446194bd04317c2779f5/users/f76215c187a24cc3813d041f836da1e9/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:44.051986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e2af3-92e1-46fd-b7f2-26f0c4b5af7a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:44.052969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e2af3-92e1-46fd-b7f2-26f0c4b5af7a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:44.053426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e2af3-92e1-46fd-b7f2-26f0c4b5af7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:44.095747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-175e2af3-92e1-46fd-b7f2-26f0c4b5af7a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:44.099067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-175e2af3-92e1-46fd-b7f2-26f0c4b5af7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:44.168053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:44.238241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 903/1823] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:44.256890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5427f-e970-454c-bfde-44aa4a6bb89f admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:44.281458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:44.423834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5427f-e970-454c-bfde-44aa4a6bb89f admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:44.438350 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e2af3-92e1-46fd-b7f2-26f0c4b5af7a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:44.446502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-175e2af3-92e1-46fd-b7f2-26f0c4b5af7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:44.452367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38013684-cbb9-4c00-bd86-cba793e8256c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:44.453440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38013684-cbb9-4c00-bd86-cba793e8256c service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:44.453730 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38013684-cbb9-4c00-bd86-cba793e8256c service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:44.454081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38013684-cbb9-4c00-bd86-cba793e8256c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:44.467788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e2af3-92e1-46fd-b7f2-26f0c4b5af7a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:44.477374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:44.478762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38013684-cbb9-4c00-bd86-cba793e8256c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:44.478889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:44.479507 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:44.480003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] PATH_INFO: `/v3/projects/deb90f37a44d47518ed4b97f213f6bb4` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:44.481551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deb90f37a44d47518ed4b97f213f6bb4)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:44.482138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38013684-cbb9-4c00-bd86-cba793e8256c service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:44.484475 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:44.509293 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 921/1824] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:44.566450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deb90f37a44d47518ed4b97f213f6bb4', 'request_id': 'req-1d327755-9ffc-4f02-812c-f8037d22d298'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:44.578566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deb90f37a44d47518ed4b97f213f6bb4', 'request_id': 'req-1d327755-9ffc-4f02-812c-f8037d22d298'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:44.593612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deb90f37a44d47518ed4b97f213f6bb4', 'request_id': 'req-1d327755-9ffc-4f02-812c-f8037d22d298'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:44.624030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] The token cache is being invalidate because project deb90f37a44d47518ed4b97f213f6bb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:44.624724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:44.626200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:44.629188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 904/1825] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:44 2022] DELETE /identity/v3/projects/deb90f37a44d47518ed4b97f213f6bb4 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:36:44.634225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38013684-cbb9-4c00-bd86-cba793e8256c service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:44.646975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38013684-cbb9-4c00-bd86-cba793e8256c service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:44.824413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f521d4-8388-40b0-9784-a3002eb19dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:44.825609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f521d4-8388-40b0-9784-a3002eb19dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:44.825907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f521d4-8388-40b0-9784-a3002eb19dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:44.826195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f521d4-8388-40b0-9784-a3002eb19dd4 admin admin] PATH_INFO: `/v3/users/7d1b9131bca24b0a929fd192d42a7ef4` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:44.828002 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f521d4-8388-40b0-9784-a3002eb19dd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d1b9131bca24b0a929fd192d42a7ef4)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:44.831124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f521d4-8388-40b0-9784-a3002eb19dd4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:44.986728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-45f521d4-8388-40b0-9784-a3002eb19dd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d1b9131bca24b0a929fd192d42a7ef4'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:44.987919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-45f521d4-8388-40b0-9784-a3002eb19dd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d1b9131bca24b0a929fd192d42a7ef4'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:45.003878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-45f521d4-8388-40b0-9784-a3002eb19dd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d1b9131bca24b0a929fd192d42a7ef4'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:45.054290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-45f521d4-8388-40b0-9784-a3002eb19dd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d1b9131bca24b0a929fd192d42a7ef4'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:45.055515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-45f521d4-8388-40b0-9784-a3002eb19dd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d1b9131bca24b0a929fd192d42a7ef4'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:45.077198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 922/1826] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:44 2022] DELETE /identity/v3/users/7d1b9131bca24b0a929fd192d42a7ef4 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:46.237388 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:46.240861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:46.274092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d327755-9ffc-4f02-812c-f8037d22d298 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:46.504936 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:46.505931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:46.506211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:46.506493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2 admin admin] PATH_INFO: `/v3/projects/4e2ddea64c7d4063aad5e64c7c21018e` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:46.522678 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e2ddea64c7d4063aad5e64c7c21018e)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:46.526099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:46.656318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e2ddea64c7d4063aad5e64c7c21018e', 'request_id': 'req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:46.675549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:46.682466 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e2ddea64c7d4063aad5e64c7c21018e', 'request_id': 'req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:46.722330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e2ddea64c7d4063aad5e64c7c21018e', 'request_id': 'req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:46.764665 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2 admin admin] The token cache is being invalidate because project 4e2ddea64c7d4063aad5e64c7c21018e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:46.774147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:46.775285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ecde5e8-f0c2-4fc8-9154-4e3c3f9aa9d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:46.779999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 905/1827] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:46 2022] DELETE /identity/v3/projects/4e2ddea64c7d4063aad5e64c7c21018e => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:46.786628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c99b1-b34a-4b24-8d16-80a4e17f21f0 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:46.787840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c99b1-b34a-4b24-8d16-80a4e17f21f0 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:46.788170 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c99b1-b34a-4b24-8d16-80a4e17f21f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:46.872518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-4b4c99b1-b34a-4b24-8d16-80a4e17f21f0 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:46.880110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4b4c99b1-b34a-4b24-8d16-80a4e17f21f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:46.957242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:47.067309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 923/1828] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:47.101009 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b9cd0-4c54-44e4-b12e-d1a55a2bf6f0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:47.104087 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b9cd0-4c54-44e4-b12e-d1a55a2bf6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:47.104648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b9cd0-4c54-44e4-b12e-d1a55a2bf6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:47.172895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-be3b9cd0-4c54-44e4-b12e-d1a55a2bf6f0 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:47.174691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-be3b9cd0-4c54-44e4-b12e-d1a55a2bf6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:47.192860 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 906/1829] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:47.194513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c99b1-b34a-4b24-8d16-80a4e17f21f0 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:47.238483 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c99b1-b34a-4b24-8d16-80a4e17f21f0 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:47.274383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6ed78-ec08-4d21-9c79-f4bb1bd97f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:47.276191 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6ed78-ec08-4d21-9c79-f4bb1bd97f0c admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:47.276641 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6ed78-ec08-4d21-9c79-f4bb1bd97f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:47.277161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6ed78-ec08-4d21-9c79-f4bb1bd97f0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:47.278295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6ed78-ec08-4d21-9c79-f4bb1bd97f0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:47.281366 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6ed78-ec08-4d21-9c79-f4bb1bd97f0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:47.283268 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6ed78-ec08-4d21-9c79-f4bb1bd97f0c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:47.329069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 924/1830] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:47.339681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b9cd0-4c54-44e4-b12e-d1a55a2bf6f0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:47.370026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b9cd0-4c54-44e4-b12e-d1a55a2bf6f0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:47.375120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcc626-284d-4ddd-8912-b357c7fb40c5 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:47.375537 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcc626-284d-4ddd-8912-b357c7fb40c5 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:47.375829 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcc626-284d-4ddd-8912-b357c7fb40c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:47.406360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729b0ea-db62-4a63-a0e5-76cff8a9bc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:47.407070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729b0ea-db62-4a63-a0e5-76cff8a9bc25 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:47.413157 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729b0ea-db62-4a63-a0e5-76cff8a9bc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:47.415347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729b0ea-db62-4a63-a0e5-76cff8a9bc25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:47.416931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729b0ea-db62-4a63-a0e5-76cff8a9bc25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:47.426359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729b0ea-db62-4a63-a0e5-76cff8a9bc25 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:47.498244 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-50dcc626-284d-4ddd-8912-b357c7fb40c5 None None] MFA Rules not processed for user `f76215c187a24cc3813d041f836da1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:47.499743 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-50dcc626-284d-4ddd-8912-b357c7fb40c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:47.505003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 907/1831] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:47 2022] POST /identity/v3/projects => generated 389 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:47.509715 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729b0ea-db62-4a63-a0e5-76cff8a9bc25 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:47.784694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 925/1832] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:47.785850 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dcc626-284d-4ddd-8912-b357c7fb40c5 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:47.788515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-50dcc626-284d-4ddd-8912-b357c7fb40c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:47.790167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729b0ea-db62-4a63-a0e5-76cff8a9bc25 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:47.829437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39c58c-c3fe-4b53-8406-e1a96a9b59c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:47.830815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dcc626-284d-4ddd-8912-b357c7fb40c5 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:47.831112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39c58c-c3fe-4b53-8406-e1a96a9b59c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:47.831810 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39c58c-c3fe-4b53-8406-e1a96a9b59c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:47.832500 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39c58c-c3fe-4b53-8406-e1a96a9b59c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:47.834934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39c58c-c3fe-4b53-8406-e1a96a9b59c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:47.837379 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b4324-9e40-47d7-b5e6-dcd995deede1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:47.838072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b4324-9e40-47d7-b5e6-dcd995deede1 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:47.838361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b4324-9e40-47d7-b5e6-dcd995deede1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:47.838644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b4324-9e40-47d7-b5e6-dcd995deede1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:47.839457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b4324-9e40-47d7-b5e6-dcd995deede1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:47.841049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39c58c-c3fe-4b53-8406-e1a96a9b59c5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:47.855536 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b4324-9e40-47d7-b5e6-dcd995deede1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:47.926233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 908/1833] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:47 2022] POST /identity/v3/users => generated 295 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:36:47.932968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39c58c-c3fe-4b53-8406-e1a96a9b59c5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:47.943981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 926/1834] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:47 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:47.963697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b4324-9e40-47d7-b5e6-dcd995deede1 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:47.980784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39c58c-c3fe-4b53-8406-e1a96a9b59c5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.082525 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb8ed5-add5-41f1-aeb0-921968756aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:48.083699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb8ed5-add5-41f1-aeb0-921968756aed admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.084081 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb8ed5-add5-41f1-aeb0-921968756aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.084458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb8ed5-add5-41f1-aeb0-921968756aed admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.085474 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb8ed5-add5-41f1-aeb0-921968756aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:48.093668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb8ed5-add5-41f1-aeb0-921968756aed admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:48.117507 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 909/1835] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:48.119613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb8ed5-add5-41f1-aeb0-921968756aed admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:48.139902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b4324-9e40-47d7-b5e6-dcd995deede1 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.146311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb8ed5-add5-41f1-aeb0-921968756aed admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.152827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb8704-b6ae-4ef9-a9dc-bfb602471f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:48.153546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb8704-b6ae-4ef9-a9dc-bfb602471f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.153827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb8704-b6ae-4ef9-a9dc-bfb602471f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.154154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb8704-b6ae-4ef9-a9dc-bfb602471f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.154282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83752903-04aa-47b7-ba0a-29e560c9cc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:48.154892 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb8704-b6ae-4ef9-a9dc-bfb602471f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:48.157152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb8704-b6ae-4ef9-a9dc-bfb602471f67 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:48.157284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83752903-04aa-47b7-ba0a-29e560c9cc0a admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.157737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83752903-04aa-47b7-ba0a-29e560c9cc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.158309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83752903-04aa-47b7-ba0a-29e560c9cc0a admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.160765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83752903-04aa-47b7-ba0a-29e560c9cc0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:48.165185 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83752903-04aa-47b7-ba0a-29e560c9cc0a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:48.170819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 910/1836] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:48.176765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb8704-b6ae-4ef9-a9dc-bfb602471f67 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:48.197732 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb8704-b6ae-4ef9-a9dc-bfb602471f67 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.208178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8aa881-a4fb-4cf6-880f-79eaf8808332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:48.210103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8aa881-a4fb-4cf6-880f-79eaf8808332 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.210392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8aa881-a4fb-4cf6-880f-79eaf8808332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.210671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8aa881-a4fb-4cf6-880f-79eaf8808332 admin admin] PATH_INFO: `/v3/projects/349ff401517241dfa528442d0a26b9df/users/2a4402392c904e51846867c351418f5a/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.216625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 927/1837] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:47 2022] POST /identity/v3/users => generated 292 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:48.218216 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83752903-04aa-47b7-ba0a-29e560c9cc0a admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:48.239434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83752903-04aa-47b7-ba0a-29e560c9cc0a admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.242696 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8aa881-a4fb-4cf6-880f-79eaf8808332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=349ff401517241dfa528442d0a26b9df, user_id=2a4402392c904e51846867c351418f5a, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:48.244866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8aa881-a4fb-4cf6-880f-79eaf8808332 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:48.247168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f06f5-d119-4ab5-8eb3-d2cb37d85ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:48.249352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f06f5-d119-4ab5-8eb3-d2cb37d85ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.250075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f06f5-d119-4ab5-8eb3-d2cb37d85ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.250390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f06f5-d119-4ab5-8eb3-d2cb37d85ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.252527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f06f5-d119-4ab5-8eb3-d2cb37d85ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:48.257531 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f06f5-d119-4ab5-8eb3-d2cb37d85ce7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:48.266731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 911/1838] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:48 2022] PUT /identity/v3/projects/349ff401517241dfa528442d0a26b9df/users/2a4402392c904e51846867c351418f5a/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:48.268124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7403d-1cf8-470a-a62d-d308c0b06e85 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.268411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7403d-1cf8-470a-a62d-d308c0b06e85 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.268687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7403d-1cf8-470a-a62d-d308c0b06e85 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.272014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 912/1839] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jun 7 08:36:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:48.284942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a4a47-ccce-42c7-a1f1-3857f819df18 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.289053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a4a47-ccce-42c7-a1f1-3857f819df18 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.289409 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a4a47-ccce-42c7-a1f1-3857f819df18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.304899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 928/1840] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:48.304899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc928961-a73c-42cb-9d06-89c78091c8c3 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.304899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc928961-a73c-42cb-9d06-89c78091c8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.304899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc928961-a73c-42cb-9d06-89c78091c8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.320515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:48.335498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:48.337064 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-dc928961-a73c-42cb-9d06-89c78091c8c3 None None] MFA Rules not processed for user `3f2a2bf913d1412f996acf4401ba7673`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:48.338550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-dc928961-a73c-42cb-9d06-89c78091c8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:48.374599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-089a4a47-ccce-42c7-a1f1-3857f819df18 None None] MFA Rules not processed for user `2a4402392c904e51846867c351418f5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:48.376828 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-089a4a47-ccce-42c7-a1f1-3857f819df18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:48.461830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 929/1841] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jun 7 08:36:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:48.463406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc928961-a73c-42cb-9d06-89c78091c8c3 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:48.477727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 913/1842] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:48.482419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a4a47-ccce-42c7-a1f1-3857f819df18 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:48.486266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-089a4a47-ccce-42c7-a1f1-3857f819df18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:48.507971 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a4a47-ccce-42c7-a1f1-3857f819df18 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.514197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dda9f5-6fe7-4369-a832-2990650c3906 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:48.514776 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dda9f5-6fe7-4369-a832-2990650c3906 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.515054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dda9f5-6fe7-4369-a832-2990650c3906 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.515326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dda9f5-6fe7-4369-a832-2990650c3906 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.525045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dda9f5-6fe7-4369-a832-2990650c3906 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:48.526694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dda9f5-6fe7-4369-a832-2990650c3906 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:48.544097 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 914/1843] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:48.545608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dda9f5-6fe7-4369-a832-2990650c3906 service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:48.551785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc928961-a73c-42cb-9d06-89c78091c8c3 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.558054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dda9f5-6fe7-4369-a832-2990650c3906 service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.559707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48c611-b4b8-4d43-9c85-b53a1a0dc02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:48.560804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48c611-b4b8-4d43-9c85-b53a1a0dc02b admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.561266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48c611-b4b8-4d43-9c85-b53a1a0dc02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.561681 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48c611-b4b8-4d43-9c85-b53a1a0dc02b admin admin] PATH_INFO: `/v3/projects/303345600487488c8856a0892ef1583a/users/ac52e3d679614ce59e368229517f8660/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.563047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5578eb-6d94-4e96-b8eb-15039a1e7217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:48.563829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5578eb-6d94-4e96-b8eb-15039a1e7217 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.564261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5578eb-6d94-4e96-b8eb-15039a1e7217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.564548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5578eb-6d94-4e96-b8eb-15039a1e7217 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.565306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5578eb-6d94-4e96-b8eb-15039a1e7217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:48.567236 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5578eb-6d94-4e96-b8eb-15039a1e7217 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:48.587497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48c611-b4b8-4d43-9c85-b53a1a0dc02b admin admin] RBAC: Authorizing `identity:create_grant(project_id=303345600487488c8856a0892ef1583a, user_id=ac52e3d679614ce59e368229517f8660, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:48.590274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48c611-b4b8-4d43-9c85-b53a1a0dc02b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:48.609608 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 930/1844] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:48 2022] PUT /identity/v3/projects/303345600487488c8856a0892ef1583a/users/ac52e3d679614ce59e368229517f8660/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:48.613127 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 915/1845] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:48 2022] POST /identity/v3/projects => generated 389 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:48.615066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48c611-b4b8-4d43-9c85-b53a1a0dc02b admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:48.615684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5578eb-6d94-4e96-b8eb-15039a1e7217 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:48.689942 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48c611-b4b8-4d43-9c85-b53a1a0dc02b admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.690872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5578eb-6d94-4e96-b8eb-15039a1e7217 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.695927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaab81-89ef-481e-9e44-d4a200f953ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:48.696798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaab81-89ef-481e-9e44-d4a200f953ba admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.697078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaab81-89ef-481e-9e44-d4a200f953ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.697410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaab81-89ef-481e-9e44-d4a200f953ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.698172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaab81-89ef-481e-9e44-d4a200f953ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:48.699188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbf137-57a2-48d1-96be-e89b26dd18d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:48.699779 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaab81-89ef-481e-9e44-d4a200f953ba admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:48.700153 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbf137-57a2-48d1-96be-e89b26dd18d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.700642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbf137-57a2-48d1-96be-e89b26dd18d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.700973 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbf137-57a2-48d1-96be-e89b26dd18d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.702261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbf137-57a2-48d1-96be-e89b26dd18d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:48.704330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbf137-57a2-48d1-96be-e89b26dd18d8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:48.711875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 931/1846] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:48.716186 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaab81-89ef-481e-9e44-d4a200f953ba admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:48.729016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaab81-89ef-481e-9e44-d4a200f953ba admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.729448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 916/1847] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:48 2022] POST /identity/v3/users => generated 301 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:36:48.733460 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbf137-57a2-48d1-96be-e89b26dd18d8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:48.733881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71b151-ddac-4fec-9b96-0991bef950c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:48.734780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71b151-ddac-4fec-9b96-0991bef950c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.735063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71b151-ddac-4fec-9b96-0991bef950c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.735371 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71b151-ddac-4fec-9b96-0991bef950c4 admin admin] PATH_INFO: `/v3/projects/303345600487488c8856a0892ef1583a/users/ac52e3d679614ce59e368229517f8660/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.737406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71b151-ddac-4fec-9b96-0991bef950c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=303345600487488c8856a0892ef1583a, user_id=ac52e3d679614ce59e368229517f8660, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:48.740836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71b151-ddac-4fec-9b96-0991bef950c4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:48.744294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbf137-57a2-48d1-96be-e89b26dd18d8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.749076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9dbf9-d25b-476f-befe-eb484e116b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:48.749773 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9dbf9-d25b-476f-befe-eb484e116b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.750062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9dbf9-d25b-476f-befe-eb484e116b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.750399 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9dbf9-d25b-476f-befe-eb484e116b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.751059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9dbf9-d25b-476f-befe-eb484e116b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:48.752751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9dbf9-d25b-476f-befe-eb484e116b95 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:48.753878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 932/1848] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:48 2022] PUT /identity/v3/projects/303345600487488c8856a0892ef1583a/users/ac52e3d679614ce59e368229517f8660/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:36:48.767692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 917/1849] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:36:48.769025 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71b151-ddac-4fec-9b96-0991bef950c4 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:48.774787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:48.859580 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71b151-ddac-4fec-9b96-0991bef950c4 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.865228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb54c68-e005-4dbe-a4db-f1b8430177cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:48.866238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb54c68-e005-4dbe-a4db-f1b8430177cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:48.866520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb54c68-e005-4dbe-a4db-f1b8430177cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:48.866791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb54c68-e005-4dbe-a4db-f1b8430177cb admin admin] PATH_INFO: `/v3/projects/127fd0790a7d449da7252ab9ef03a113/users/793e2677ddae4524b192a9d72c4e48f0/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:48.884067 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb54c68-e005-4dbe-a4db-f1b8430177cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=127fd0790a7d449da7252ab9ef03a113, user_id=793e2677ddae4524b192a9d72c4e48f0, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:48.886865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb54c68-e005-4dbe-a4db-f1b8430177cb admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:48.908522 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 933/1850] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:48 2022] PUT /identity/v3/projects/127fd0790a7d449da7252ab9ef03a113/users/793e2677ddae4524b192a9d72c4e48f0/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:48.913492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9dbf9-d25b-476f-befe-eb484e116b95 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:48.926148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9dbf9-d25b-476f-befe-eb484e116b95 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:48.986661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:49.028261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be95f36-b2dc-40ec-812a-ddf94aa2122e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:49.028989 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be95f36-b2dc-40ec-812a-ddf94aa2122e admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:49.029353 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be95f36-b2dc-40ec-812a-ddf94aa2122e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:49.029718 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be95f36-b2dc-40ec-812a-ddf94aa2122e admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:49.030590 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be95f36-b2dc-40ec-812a-ddf94aa2122e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:49.032327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be95f36-b2dc-40ec-812a-ddf94aa2122e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:49.046003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 918/1851] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:49.050018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb54c68-e005-4dbe-a4db-f1b8430177cb admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:49.062888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb54c68-e005-4dbe-a4db-f1b8430177cb admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:49.068073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3433e5-c1a9-4e24-bf56-4fdc147b7233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:49.069049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3433e5-c1a9-4e24-bf56-4fdc147b7233 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:49.069439 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3433e5-c1a9-4e24-bf56-4fdc147b7233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:49.069799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3433e5-c1a9-4e24-bf56-4fdc147b7233 admin admin] PATH_INFO: `/v3/projects/127fd0790a7d449da7252ab9ef03a113/users/793e2677ddae4524b192a9d72c4e48f0/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:49.072925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3433e5-c1a9-4e24-bf56-4fdc147b7233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=127fd0790a7d449da7252ab9ef03a113, user_id=793e2677ddae4524b192a9d72c4e48f0, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:49.075918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3433e5-c1a9-4e24-bf56-4fdc147b7233 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:49.092205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 934/1852] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:49 2022] PUT /identity/v3/projects/127fd0790a7d449da7252ab9ef03a113/users/793e2677ddae4524b192a9d72c4e48f0/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:49.103340 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c6df2-f853-4621-8d6f-20e68445e5fd None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:49.103807 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c6df2-f853-4621-8d6f-20e68445e5fd None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:49.104248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c6df2-f853-4621-8d6f-20e68445e5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:49.140201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-3e1c6df2-f853-4621-8d6f-20e68445e5fd None None] MFA Rules not processed for user `793e2677ddae4524b192a9d72c4e48f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:49.142443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3e1c6df2-f853-4621-8d6f-20e68445e5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:49.274874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 919/1853] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:49.298738 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3433e5-c1a9-4e24-bf56-4fdc147b7233 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:49.302248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ea3433e5-c1a9-4e24-bf56-4fdc147b7233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:49.405280 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3433e5-c1a9-4e24-bf56-4fdc147b7233 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:49.409036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280af6c-53d2-4ee1-81fb-6f22129d0460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:49.409733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280af6c-53d2-4ee1-81fb-6f22129d0460 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:49.410023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280af6c-53d2-4ee1-81fb-6f22129d0460 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:49.410293 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280af6c-53d2-4ee1-81fb-6f22129d0460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:49.424486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280af6c-53d2-4ee1-81fb-6f22129d0460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:49.426009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280af6c-53d2-4ee1-81fb-6f22129d0460 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:49.445238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 935/1854] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:49.619932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c6df2-f853-4621-8d6f-20e68445e5fd None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:49.748976 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc7b95-4cb8-408a-9d2e-0dee5f2fc674 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:49.749547 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc7b95-4cb8-408a-9d2e-0dee5f2fc674 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:49.750060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc7b95-4cb8-408a-9d2e-0dee5f2fc674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:49.758271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c6df2-f853-4621-8d6f-20e68445e5fd None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:49.763790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4402fd-2f13-4e05-99ce-b7f0927f2c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:49.765109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4402fd-2f13-4e05-99ce-b7f0927f2c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:49.765492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4402fd-2f13-4e05-99ce-b7f0927f2c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:49.766019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4402fd-2f13-4e05-99ce-b7f0927f2c20 admin admin] PATH_INFO: `/v3/users/2a4402392c904e51846867c351418f5a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:49.767383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4402fd-2f13-4e05-99ce-b7f0927f2c20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a4402392c904e51846867c351418f5a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:49.769449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4402fd-2f13-4e05-99ce-b7f0927f2c20 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:49.808977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-99bc7b95-4cb8-408a-9d2e-0dee5f2fc674 None None] MFA Rules not processed for user `ac52e3d679614ce59e368229517f8660`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:49.812322 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-99bc7b95-4cb8-408a-9d2e-0dee5f2fc674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:49.887146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-bc4402fd-2f13-4e05-99ce-b7f0927f2c20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a4402392c904e51846867c351418f5a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:49.887912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-bc4402fd-2f13-4e05-99ce-b7f0927f2c20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a4402392c904e51846867c351418f5a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:49.899279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-bc4402fd-2f13-4e05-99ce-b7f0927f2c20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a4402392c904e51846867c351418f5a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:49.900597 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-bc4402fd-2f13-4e05-99ce-b7f0927f2c20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a4402392c904e51846867c351418f5a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:49.914014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:49.924625 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-bc4402fd-2f13-4e05-99ce-b7f0927f2c20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a4402392c904e51846867c351418f5a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:49.963696 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 920/1855] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:49 2022] DELETE /identity/v3/users/2a4402392c904e51846867c351418f5a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:49.964013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:49.967971 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065d5fa-008c-4e9a-8c53-40c4ce5750fc None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:49.967971 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065d5fa-008c-4e9a-8c53-40c4ce5750fc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:49.969448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065d5fa-008c-4e9a-8c53-40c4ce5750fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:49.969650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 936/1856] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:49 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:49.983278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc7b95-4cb8-408a-9d2e-0dee5f2fc674 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:49.984860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-99bc7b95-4cb8-408a-9d2e-0dee5f2fc674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:50.001290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc7b95-4cb8-408a-9d2e-0dee5f2fc674 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:50.007551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-8065d5fa-008c-4e9a-8c53-40c4ce5750fc None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:50.010729 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8065d5fa-008c-4e9a-8c53-40c4ce5750fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:50.064220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec6e7f-02c6-49ab-8754-ceedc3611cb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:50.065422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec6e7f-02c6-49ab-8754-ceedc3611cb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:50.065714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec6e7f-02c6-49ab-8754-ceedc3611cb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:50.065996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec6e7f-02c6-49ab-8754-ceedc3611cb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:50.068202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-57ec6e7f-02c6-49ab-8754-ceedc3611cb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:50.086955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 921/1857] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:50.097832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065d5fa-008c-4e9a-8c53-40c4ce5750fc None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:50.101043 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8065d5fa-008c-4e9a-8c53-40c4ce5750fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:50.143210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec6e7f-02c6-49ab-8754-ceedc3611cb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:50.144799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec6e7f-02c6-49ab-8754-ceedc3611cb3 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:50.157216 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 937/1858] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:36:50.172416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065d5fa-008c-4e9a-8c53-40c4ce5750fc None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:50.176625 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22809b8a-b831-423e-8be2-f1b890b9e04a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:50.177536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22809b8a-b831-423e-8be2-f1b890b9e04a service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:50.178065 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22809b8a-b831-423e-8be2-f1b890b9e04a service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:50.178352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22809b8a-b831-423e-8be2-f1b890b9e04a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:50.189323 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22809b8a-b831-423e-8be2-f1b890b9e04a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:50.191685 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22809b8a-b831-423e-8be2-f1b890b9e04a service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:50.204895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 922/1859] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:51.759489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec6e7f-02c6-49ab-8754-ceedc3611cb3 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:51.762682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-57ec6e7f-02c6-49ab-8754-ceedc3611cb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:51.793404 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec6e7f-02c6-49ab-8754-ceedc3611cb3 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:51.798313 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be741531-3cc4-40a5-82cb-0825b70bc531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:51.798986 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be741531-3cc4-40a5-82cb-0825b70bc531 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:51.799299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be741531-3cc4-40a5-82cb-0825b70bc531 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:51.799653 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be741531-3cc4-40a5-82cb-0825b70bc531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:51.834527 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22809b8a-b831-423e-8be2-f1b890b9e04a service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:51.836829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-22809b8a-b831-423e-8be2-f1b890b9e04a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:51.881675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22809b8a-b831-423e-8be2-f1b890b9e04a service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:51.888907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:51.889998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:51.890383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:51.890763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] PATH_INFO: `/v3/projects/349ff401517241dfa528442d0a26b9df` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:51.892448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=349ff401517241dfa528442d0a26b9df)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:51.897336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:51.995777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be741531-3cc4-40a5-82cb-0825b70bc531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:51.997470 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be741531-3cc4-40a5-82cb-0825b70bc531 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:52.010934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '349ff401517241dfa528442d0a26b9df', 'request_id': 'req-c61b1323-4469-4e72-86de-177f7e7dca3b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:52.020340 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '349ff401517241dfa528442d0a26b9df', 'request_id': 'req-c61b1323-4469-4e72-86de-177f7e7dca3b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:52.032275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 938/1860] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:51 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:52.047185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '349ff401517241dfa528442d0a26b9df', 'request_id': 'req-c61b1323-4469-4e72-86de-177f7e7dca3b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:52.082352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] The token cache is being invalidate because project 349ff401517241dfa528442d0a26b9df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:52.082992 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:52.083854 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:52.085842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 923/1861] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:51 2022] DELETE /identity/v3/projects/349ff401517241dfa528442d0a26b9df => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:52.090553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be741531-3cc4-40a5-82cb-0825b70bc531 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:52.126268 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be741531-3cc4-40a5-82cb-0825b70bc531 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:52.254271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:52.280288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9276591-3802-4368-9684-bd54b076f10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:52.281219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9276591-3802-4368-9684-bd54b076f10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:52.281503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9276591-3802-4368-9684-bd54b076f10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:52.281777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9276591-3802-4368-9684-bd54b076f10b admin admin] PATH_INFO: `/v3/users/793e2677ddae4524b192a9d72c4e48f0` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:52.290420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9276591-3802-4368-9684-bd54b076f10b admin admin] RBAC: Authorizing `identity:delete_user(user_id=793e2677ddae4524b192a9d72c4e48f0)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:52.292541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9276591-3802-4368-9684-bd54b076f10b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:52.390651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c9276591-3802-4368-9684-bd54b076f10b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793e2677ddae4524b192a9d72c4e48f0'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:52.392005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c9276591-3802-4368-9684-bd54b076f10b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793e2677ddae4524b192a9d72c4e48f0'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:52.419528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c9276591-3802-4368-9684-bd54b076f10b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793e2677ddae4524b192a9d72c4e48f0'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:52.463532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c9276591-3802-4368-9684-bd54b076f10b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793e2677ddae4524b192a9d72c4e48f0'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:52.464739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c9276591-3802-4368-9684-bd54b076f10b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793e2677ddae4524b192a9d72c4e48f0'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:52.516300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 939/1862] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:52 2022] DELETE /identity/v3/users/793e2677ddae4524b192a9d72c4e48f0 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:53.823635 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:53.825276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:53.842866 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e739f8-bbbe-49f4-81b7-0d92a7ef7f03 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:53.843538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e739f8-bbbe-49f4-81b7-0d92a7ef7f03 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:53.843779 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e739f8-bbbe-49f4-81b7-0d92a7ef7f03 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:53.845846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 940/1863] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jun 7 08:36:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:53.856395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b1323-4469-4e72-86de-177f7e7dca3b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:53.863695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7cfdd-e381-47a2-be18-b1dd99656f02 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:53.863999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7cfdd-e381-47a2-be18-b1dd99656f02 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:53.864309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7cfdd-e381-47a2-be18-b1dd99656f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:53.878562 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:53.933018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1ad7cfdd-e381-47a2-be18-b1dd99656f02 None None] MFA Rules not processed for user `3f2a2bf913d1412f996acf4401ba7673`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:53.936958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1ad7cfdd-e381-47a2-be18-b1dd99656f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:54.111604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:54.113453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:54.115836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:54.116327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] PATH_INFO: `/v3/projects/127fd0790a7d449da7252ab9ef03a113` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:54.120022 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=127fd0790a7d449da7252ab9ef03a113)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:54.134895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:54.223096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '127fd0790a7d449da7252ab9ef03a113', 'request_id': 'req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:54.223375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 941/1864] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jun 7 08:36:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:54.231626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '127fd0790a7d449da7252ab9ef03a113', 'request_id': 'req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:54.248575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '127fd0790a7d449da7252ab9ef03a113', 'request_id': 'req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:54.286377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] The token cache is being invalidate because project 127fd0790a7d449da7252ab9ef03a113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:54.286696 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:54.290904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:54.292929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 924/1865] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:53 2022] DELETE /identity/v3/projects/127fd0790a7d449da7252ab9ef03a113 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:54.298089 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3456561-f895-42a8-aa7a-b7e6c1003425 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:54.298287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3456561-f895-42a8-aa7a-b7e6c1003425 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:54.298748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3456561-f895-42a8-aa7a-b7e6c1003425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:54.324532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-b3456561-f895-42a8-aa7a-b7e6c1003425 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:54.326989 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:54.328681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:54.331072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b3456561-f895-42a8-aa7a-b7e6c1003425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:54.420832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:54.439424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 942/1866] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:54.450507 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cd1cb-a04a-44a7-a26b-e2534cc9eb66 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:54.451330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cd1cb-a04a-44a7-a26b-e2534cc9eb66 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:54.451760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cd1cb-a04a-44a7-a26b-e2534cc9eb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:54.452139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d4aa5-2bc4-4f4d-89f6-d57395aafc8e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:54.460085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d758bd-4af5-4d27-b40c-38d2a5aee114 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:54.460840 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d758bd-4af5-4d27-b40c-38d2a5aee114 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:54.461230 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d758bd-4af5-4d27-b40c-38d2a5aee114 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:54.461579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d758bd-4af5-4d27-b40c-38d2a5aee114 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:54.463595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-43d758bd-4af5-4d27-b40c-38d2a5aee114 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:54.477871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-799cd1cb-a04a-44a7-a26b-e2534cc9eb66 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:54.479559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-799cd1cb-a04a-44a7-a26b-e2534cc9eb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:54.482030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d758bd-4af5-4d27-b40c-38d2a5aee114 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:54.484299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d758bd-4af5-4d27-b40c-38d2a5aee114 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:54.490824 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 943/1867] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:54.495275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cd1cb-a04a-44a7-a26b-e2534cc9eb66 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:54.499941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 925/1868] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:54.505826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cd1cb-a04a-44a7-a26b-e2534cc9eb66 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:54.510290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cc251-f73d-4c97-b9e1-62fca2c08ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:54.511007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cc251-f73d-4c97-b9e1-62fca2c08ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:54.511298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cc251-f73d-4c97-b9e1-62fca2c08ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:54.511749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cc251-f73d-4c97-b9e1-62fca2c08ea1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:54.512487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5cc251-f73d-4c97-b9e1-62fca2c08ea1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:54.512791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5cc251-f73d-4c97-b9e1-62fca2c08ea1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:54.514493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5cc251-f73d-4c97-b9e1-62fca2c08ea1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:54.531729 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 944/1869] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:36:54.532913 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d758bd-4af5-4d27-b40c-38d2a5aee114 service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:54.546033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d758bd-4af5-4d27-b40c-38d2a5aee114 service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:54.553358 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68672ef5-696c-483d-8133-52cd92af95bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:54.561556 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68672ef5-696c-483d-8133-52cd92af95bd admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:54.561980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68672ef5-696c-483d-8133-52cd92af95bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:54.562333 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68672ef5-696c-483d-8133-52cd92af95bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:54.563150 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68672ef5-696c-483d-8133-52cd92af95bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:54.565291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68672ef5-696c-483d-8133-52cd92af95bd admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:54.588623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:54.605976 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cc251-f73d-4c97-b9e1-62fca2c08ea1 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:54.606252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 926/1870] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:54 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:36:54.623146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cc251-f73d-4c97-b9e1-62fca2c08ea1 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:54.684231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030a60d-99aa-4b83-a0ca-49a0ef800a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:54.684905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030a60d-99aa-4b83-a0ca-49a0ef800a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:54.685222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030a60d-99aa-4b83-a0ca-49a0ef800a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:54.685530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030a60d-99aa-4b83-a0ca-49a0ef800a81 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:54.686226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030a60d-99aa-4b83-a0ca-49a0ef800a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:54.688141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030a60d-99aa-4b83-a0ca-49a0ef800a81 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:54.711801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 945/1871] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:54 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:54.718374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68672ef5-696c-483d-8133-52cd92af95bd admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:54.732534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68672ef5-696c-483d-8133-52cd92af95bd admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:54.741078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c0691-f085-4096-ba80-b9db60f1cc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:54.741809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c0691-f085-4096-ba80-b9db60f1cc90 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:54.742089 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c0691-f085-4096-ba80-b9db60f1cc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:54.742364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c0691-f085-4096-ba80-b9db60f1cc90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:54.743174 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c0691-f085-4096-ba80-b9db60f1cc90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:54.744789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c0691-f085-4096-ba80-b9db60f1cc90 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:54.756924 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 927/1872] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:54.760922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030a60d-99aa-4b83-a0ca-49a0ef800a81 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:54.770969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030a60d-99aa-4b83-a0ca-49a0ef800a81 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:54.775650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa396a5-4537-445a-a6b9-4d39436e04db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:54.776301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa396a5-4537-445a-a6b9-4d39436e04db admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:54.776583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa396a5-4537-445a-a6b9-4d39436e04db admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:54.776860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa396a5-4537-445a-a6b9-4d39436e04db admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:54.777613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa396a5-4537-445a-a6b9-4d39436e04db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:54.779195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa396a5-4537-445a-a6b9-4d39436e04db admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:54.789652 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 946/1873] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:54.793997 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c0691-f085-4096-ba80-b9db60f1cc90 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:54.805880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c0691-f085-4096-ba80-b9db60f1cc90 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:54.810558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c45e17-7dc5-4fbc-9554-775fb21a812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:54.811433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c45e17-7dc5-4fbc-9554-775fb21a812f admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:54.811718 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c45e17-7dc5-4fbc-9554-775fb21a812f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:54.811995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c45e17-7dc5-4fbc-9554-775fb21a812f admin admin] PATH_INFO: `/v3/projects/2caeed8ba9ec41748d056c5c61d4a955/users/a090d422255a4cc6979783381dbf1c91/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:54.827069 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c45e17-7dc5-4fbc-9554-775fb21a812f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2caeed8ba9ec41748d056c5c61d4a955, user_id=a090d422255a4cc6979783381dbf1c91, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:54.829807 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c45e17-7dc5-4fbc-9554-775fb21a812f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:54.847006 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 928/1874] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:54 2022] PUT /identity/v3/projects/2caeed8ba9ec41748d056c5c61d4a955/users/a090d422255a4cc6979783381dbf1c91/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:54.859834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da97b7b-94c3-4227-8b77-07db5bd843c3 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:54.860416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da97b7b-94c3-4227-8b77-07db5bd843c3 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:54.860540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da97b7b-94c3-4227-8b77-07db5bd843c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:54.898528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-6da97b7b-94c3-4227-8b77-07db5bd843c3 None None] MFA Rules not processed for user `a090d422255a4cc6979783381dbf1c91`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:54.900034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6da97b7b-94c3-4227-8b77-07db5bd843c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:54.967006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 947/1875] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:54.978818 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c45e17-7dc5-4fbc-9554-775fb21a812f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:54.980535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-33c45e17-7dc5-4fbc-9554-775fb21a812f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:55.041547 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c45e17-7dc5-4fbc-9554-775fb21a812f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:55.045362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2697c-c076-4217-92c2-d51fddb63b8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:55.046020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2697c-c076-4217-92c2-d51fddb63b8a service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:55.046299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2697c-c076-4217-92c2-d51fddb63b8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:55.046583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2697c-c076-4217-92c2-d51fddb63b8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:55.056537 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2697c-c076-4217-92c2-d51fddb63b8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:55.057996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2697c-c076-4217-92c2-d51fddb63b8a service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:55.067237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 929/1876] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:55.088784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da97b7b-94c3-4227-8b77-07db5bd843c3 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:55.143256 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:55.160333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da97b7b-94c3-4227-8b77-07db5bd843c3 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:55.167078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:55.168066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:55.168863 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:55.169200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] PATH_INFO: `/v3/users/a090d422255a4cc6979783381dbf1c91` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:55.170523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a090d422255a4cc6979783381dbf1c91)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:55.172529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:55.241403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a090d422255a4cc6979783381dbf1c91'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:55.242142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a090d422255a4cc6979783381dbf1c91'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:55.251483 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a090d422255a4cc6979783381dbf1c91'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:55.266376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a090d422255a4cc6979783381dbf1c91'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:55.267084 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a090d422255a4cc6979783381dbf1c91'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:55.284673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 948/1877] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:36:55 2022] DELETE /identity/v3/users/a090d422255a4cc6979783381dbf1c91 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:55.290589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02d6f4-c18f-4e77-a64f-82d97522b9ae None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:55.290874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02d6f4-c18f-4e77-a64f-82d97522b9ae None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:55.291147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02d6f4-c18f-4e77-a64f-82d97522b9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:55.309309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0c02d6f4-c18f-4e77-a64f-82d97522b9ae None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:55.310598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0c02d6f4-c18f-4e77-a64f-82d97522b9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:55.420912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 930/1878] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:55.433807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:55.435263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:55.510420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3521da-7fec-492e-b0fe-6ff810963837 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:55.514902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8824e49-dc00-4aa2-9f89-0deac3878717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:55.515577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8824e49-dc00-4aa2-9f89-0deac3878717 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:55.515860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8824e49-dc00-4aa2-9f89-0deac3878717 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:55.516150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8824e49-dc00-4aa2-9f89-0deac3878717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:55.526899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8824e49-dc00-4aa2-9f89-0deac3878717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:55.528372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8824e49-dc00-4aa2-9f89-0deac3878717 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:55.543145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 949/1879] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:56.048004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02d6f4-c18f-4e77-a64f-82d97522b9ae None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:56.049508 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0c02d6f4-c18f-4e77-a64f-82d97522b9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:56.062269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02d6f4-c18f-4e77-a64f-82d97522b9ae None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:56.066911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5a439-5848-4f26-b5ce-a1974bcd25e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:56.069449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5a439-5848-4f26-b5ce-a1974bcd25e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:56.069449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5a439-5848-4f26-b5ce-a1974bcd25e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:56.069680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5a439-5848-4f26-b5ce-a1974bcd25e3 admin admin] PATH_INFO: `/v3/projects/2caeed8ba9ec41748d056c5c61d4a955` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:56.070771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5a439-5848-4f26-b5ce-a1974bcd25e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2caeed8ba9ec41748d056c5c61d4a955)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:56.072825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5a439-5848-4f26-b5ce-a1974bcd25e3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:56.138961 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-08f5a439-5848-4f26-b5ce-a1974bcd25e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2caeed8ba9ec41748d056c5c61d4a955', 'request_id': 'req-08f5a439-5848-4f26-b5ce-a1974bcd25e3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:56.144755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-08f5a439-5848-4f26-b5ce-a1974bcd25e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2caeed8ba9ec41748d056c5c61d4a955', 'request_id': 'req-08f5a439-5848-4f26-b5ce-a1974bcd25e3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:56.148952 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:56.153255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-08f5a439-5848-4f26-b5ce-a1974bcd25e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2caeed8ba9ec41748d056c5c61d4a955', 'request_id': 'req-08f5a439-5848-4f26-b5ce-a1974bcd25e3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:56.166261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-08f5a439-5848-4f26-b5ce-a1974bcd25e3 admin admin] The token cache is being invalidate because project 2caeed8ba9ec41748d056c5c61d4a955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:36:56.166551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-08f5a439-5848-4f26-b5ce-a1974bcd25e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:56.167211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-08f5a439-5848-4f26-b5ce-a1974bcd25e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:36:56.168760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 931/1880] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:36:56 2022] DELETE /identity/v3/projects/2caeed8ba9ec41748d056c5c61d4a955 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:56.173866 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7d49a-885b-4f8a-a2be-3fb43ec336a1 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:56.174334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7d49a-885b-4f8a-a2be-3fb43ec336a1 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:56.174442 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7d49a-885b-4f8a-a2be-3fb43ec336a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:56.192511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-9fa7d49a-885b-4f8a-a2be-3fb43ec336a1 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:56.194270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9fa7d49a-885b-4f8a-a2be-3fb43ec336a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:56.259900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 950/1881] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:56.270968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b944539-3473-44ec-9398-8ccb4554fa04 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:56.271330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b944539-3473-44ec-9398-8ccb4554fa04 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:56.271701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b944539-3473-44ec-9398-8ccb4554fa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:56.291300 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6b944539-3473-44ec-9398-8ccb4554fa04 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:56.292610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6b944539-3473-44ec-9398-8ccb4554fa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:56.301243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 932/1882] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:56.306712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7d49a-885b-4f8a-a2be-3fb43ec336a1 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:56.316467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7d49a-885b-4f8a-a2be-3fb43ec336a1 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:56.321376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b37ac-147e-451d-ab05-f4838c8dfcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:56.322026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b37ac-147e-451d-ab05-f4838c8dfcd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:56.322314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b37ac-147e-451d-ab05-f4838c8dfcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:56.322594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b37ac-147e-451d-ab05-f4838c8dfcd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:56.323325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b37ac-147e-451d-ab05-f4838c8dfcd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:36:56.323623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b37ac-147e-451d-ab05-f4838c8dfcd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:56.325285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b37ac-147e-451d-ab05-f4838c8dfcd7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:56.332093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:56.341426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 951/1883] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:56.342680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b944539-3473-44ec-9398-8ccb4554fa04 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:56.353553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b944539-3473-44ec-9398-8ccb4554fa04 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:56.359030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa3988-7730-442b-82dc-1eddb89cbea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:56.359882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa3988-7730-442b-82dc-1eddb89cbea0 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:56.360199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa3988-7730-442b-82dc-1eddb89cbea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:56.360646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa3988-7730-442b-82dc-1eddb89cbea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:56.361296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa3988-7730-442b-82dc-1eddb89cbea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:56.363080 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa3988-7730-442b-82dc-1eddb89cbea0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:56.391922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 933/1884] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:56 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:36:56.396023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b37ac-147e-451d-ab05-f4838c8dfcd7 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:56.406338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b37ac-147e-451d-ab05-f4838c8dfcd7 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:56.474499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5701f8-4fe8-4ef2-8f17-c18d341bb3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:56.475220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5701f8-4fe8-4ef2-8f17-c18d341bb3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:56.475499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5701f8-4fe8-4ef2-8f17-c18d341bb3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:56.476108 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5701f8-4fe8-4ef2-8f17-c18d341bb3c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:56.476598 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5701f8-4fe8-4ef2-8f17-c18d341bb3c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:56.478791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5701f8-4fe8-4ef2-8f17-c18d341bb3c9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:56.517040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 952/1885] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:56 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:56.518294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa3988-7730-442b-82dc-1eddb89cbea0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:56.530502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa3988-7730-442b-82dc-1eddb89cbea0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:56.540796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d8333-7677-40c9-a2e2-46cd57881820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:56.541925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d8333-7677-40c9-a2e2-46cd57881820 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:56.542394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d8333-7677-40c9-a2e2-46cd57881820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:56.542513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d8333-7677-40c9-a2e2-46cd57881820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:56.543410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d8333-7677-40c9-a2e2-46cd57881820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:56.545598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d8333-7677-40c9-a2e2-46cd57881820 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:56.587710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5701f8-4fe8-4ef2-8f17-c18d341bb3c9 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:56.593186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 934/1886] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:36:56.608172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5701f8-4fe8-4ef2-8f17-c18d341bb3c9 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:56.624144 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a92fd-3656-4e92-86f4-e8ad638c40dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:56.625893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a92fd-3656-4e92-86f4-e8ad638c40dc admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:56.626712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a92fd-3656-4e92-86f4-e8ad638c40dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:56.627362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a92fd-3656-4e92-86f4-e8ad638c40dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:56.629369 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a92fd-3656-4e92-86f4-e8ad638c40dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:56.634643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a92fd-3656-4e92-86f4-e8ad638c40dc admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:56.659518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 953/1887] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:56.665524 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d8333-7677-40c9-a2e2-46cd57881820 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:56.686135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d8333-7677-40c9-a2e2-46cd57881820 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:56.695878 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ed68f-087e-4db8-bd77-bca31873c6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:56.697488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ed68f-087e-4db8-bd77-bca31873c6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:56.697973 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ed68f-087e-4db8-bd77-bca31873c6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:56.698467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ed68f-087e-4db8-bd77-bca31873c6fa admin admin] PATH_INFO: `/v3/projects/36eadc7b786244a7a5021e55ba7c3068/users/b82e8e695d5745128e6ac6ff8198b163/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:56.728988 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ed68f-087e-4db8-bd77-bca31873c6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=36eadc7b786244a7a5021e55ba7c3068, user_id=b82e8e695d5745128e6ac6ff8198b163, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:56.742024 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ed68f-087e-4db8-bd77-bca31873c6fa admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:56.769219 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 935/1888] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:56 2022] PUT /identity/v3/projects/36eadc7b786244a7a5021e55ba7c3068/users/b82e8e695d5745128e6ac6ff8198b163/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:56.782197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8e308-594f-46ee-b60d-9c3b2dde65b1 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:56.783183 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8e308-594f-46ee-b60d-9c3b2dde65b1 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:56.783829 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8e308-594f-46ee-b60d-9c3b2dde65b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:56.837591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-b8f8e308-594f-46ee-b60d-9c3b2dde65b1 None None] MFA Rules not processed for user `b82e8e695d5745128e6ac6ff8198b163`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:56.839332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b8f8e308-594f-46ee-b60d-9c3b2dde65b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:56.944984 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 954/1889] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:56.950712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ed68f-087e-4db8-bd77-bca31873c6fa admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:57.044518 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ed68f-087e-4db8-bd77-bca31873c6fa admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:57.054348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738b5c5-ddd8-4853-8d3e-00354d29615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:57.056000 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738b5c5-ddd8-4853-8d3e-00354d29615a admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:57.056554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738b5c5-ddd8-4853-8d3e-00354d29615a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:57.057188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738b5c5-ddd8-4853-8d3e-00354d29615a admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:57.058428 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738b5c5-ddd8-4853-8d3e-00354d29615a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:57.060854 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738b5c5-ddd8-4853-8d3e-00354d29615a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:57.109655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 936/1890] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:56 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:57.113878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8e308-594f-46ee-b60d-9c3b2dde65b1 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:57.132414 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8e308-594f-46ee-b60d-9c3b2dde65b1 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:57.225784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4dea54-ff7e-4374-8565-f3123ecfdc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:57.226485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dea54-ff7e-4374-8565-f3123ecfdc3f admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:57.226749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dea54-ff7e-4374-8565-f3123ecfdc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:57.227020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dea54-ff7e-4374-8565-f3123ecfdc3f admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:57.227876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4dea54-ff7e-4374-8565-f3123ecfdc3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:57.229877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4dea54-ff7e-4374-8565-f3123ecfdc3f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:57.242759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:57.259635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 955/1891] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:57 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:36:57.263367 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738b5c5-ddd8-4853-8d3e-00354d29615a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:57.276996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738b5c5-ddd8-4853-8d3e-00354d29615a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:57.283427 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81081950-ed41-4b26-8369-7565d7d44676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:57.284299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81081950-ed41-4b26-8369-7565d7d44676 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:57.284722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81081950-ed41-4b26-8369-7565d7d44676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:57.285130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81081950-ed41-4b26-8369-7565d7d44676 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:57.286253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81081950-ed41-4b26-8369-7565d7d44676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:57.288330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81081950-ed41-4b26-8369-7565d7d44676 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:57.300395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 937/1892] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:36:57.304601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4dea54-ff7e-4374-8565-f3123ecfdc3f admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:57.319168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4dea54-ff7e-4374-8565-f3123ecfdc3f admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:57.326100 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440895e-f3f8-4d99-8da0-e529e99137a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:57.326892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440895e-f3f8-4d99-8da0-e529e99137a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:57.327700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440895e-f3f8-4d99-8da0-e529e99137a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:57.327700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440895e-f3f8-4d99-8da0-e529e99137a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:57.328470 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440895e-f3f8-4d99-8da0-e529e99137a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:57.330350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440895e-f3f8-4d99-8da0-e529e99137a5 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:57.350082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 956/1893] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:57.356676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81081950-ed41-4b26-8369-7565d7d44676 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:57.370237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81081950-ed41-4b26-8369-7565d7d44676 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:57.376268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b6e85-157d-434f-b80b-90a9f12328ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:57.377475 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b6e85-157d-434f-b80b-90a9f12328ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:57.377932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b6e85-157d-434f-b80b-90a9f12328ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:57.378374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b6e85-157d-434f-b80b-90a9f12328ff admin admin] PATH_INFO: `/v3/projects/7a1e192c2d344074b8530f27e6538c5a/users/cfdceb75d4844ea19f8de36c12796100/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:57.396900 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b6e85-157d-434f-b80b-90a9f12328ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1e192c2d344074b8530f27e6538c5a, user_id=cfdceb75d4844ea19f8de36c12796100, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:57.400088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b6e85-157d-434f-b80b-90a9f12328ff admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:57.422500 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 938/1894] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:57 2022] PUT /identity/v3/projects/7a1e192c2d344074b8530f27e6538c5a/users/cfdceb75d4844ea19f8de36c12796100/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:36:57.435540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436f08c-8932-4114-a23e-27b868336c12 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:57.436386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436f08c-8932-4114-a23e-27b868336c12 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:57.437637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436f08c-8932-4114-a23e-27b868336c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:57.495370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-6436f08c-8932-4114-a23e-27b868336c12 None None] MFA Rules not processed for user `cfdceb75d4844ea19f8de36c12796100`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:57.496710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6436f08c-8932-4114-a23e-27b868336c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:57.575919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 957/1895] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:57.580798 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b6e85-157d-434f-b80b-90a9f12328ff admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:57.634712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:57.662142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b6e85-157d-434f-b80b-90a9f12328ff admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:57.667783 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ae670-b03e-4587-b69d-f76e02f0b123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:57.668413 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ae670-b03e-4587-b69d-f76e02f0b123 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:57.668700 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ae670-b03e-4587-b69d-f76e02f0b123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:57.668981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ae670-b03e-4587-b69d-f76e02f0b123 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:57.669704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ae670-b03e-4587-b69d-f76e02f0b123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:57.671600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ae670-b03e-4587-b69d-f76e02f0b123 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:57.700605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 939/1896] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:36:57 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:57.701752 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436f08c-8932-4114-a23e-27b868336c12 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:57.716748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436f08c-8932-4114-a23e-27b868336c12 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:57.780605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43878194-2034-4070-89b9-1c6beba65a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:57.781435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43878194-2034-4070-89b9-1c6beba65a4e admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:57.781772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43878194-2034-4070-89b9-1c6beba65a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:57.782083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43878194-2034-4070-89b9-1c6beba65a4e admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:57.782909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43878194-2034-4070-89b9-1c6beba65a4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:57.785302 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43878194-2034-4070-89b9-1c6beba65a4e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:57.811194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 958/1897] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:36:57 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:36:57.815167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ae670-b03e-4587-b69d-f76e02f0b123 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:57.826892 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ae670-b03e-4587-b69d-f76e02f0b123 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:57.832055 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce777bab-1021-4680-8954-2af4d663eca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:57.833742 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce777bab-1021-4680-8954-2af4d663eca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:57.834054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce777bab-1021-4680-8954-2af4d663eca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:57.834392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce777bab-1021-4680-8954-2af4d663eca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:57.835217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce777bab-1021-4680-8954-2af4d663eca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:57.836967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce777bab-1021-4680-8954-2af4d663eca2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:57.850855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 940/1898] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:57.857008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43878194-2034-4070-89b9-1c6beba65a4e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:57.867732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43878194-2034-4070-89b9-1c6beba65a4e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:57.874888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d128b-8730-4ac6-a9a4-c223d3865dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:57.875794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d128b-8730-4ac6-a9a4-c223d3865dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:57.876068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d128b-8730-4ac6-a9a4-c223d3865dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:57.876349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d128b-8730-4ac6-a9a4-c223d3865dc8 admin admin] PATH_INFO: `/v3/projects/449be8dc1626483c9c9f2b4a06aae479/users/0814d7e95f64451f97fa436fda771fbb/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:57.912967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d128b-8730-4ac6-a9a4-c223d3865dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=449be8dc1626483c9c9f2b4a06aae479, user_id=0814d7e95f64451f97fa436fda771fbb, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:57.916184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d128b-8730-4ac6-a9a4-c223d3865dc8 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:57.941261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 959/1899] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:57 2022] PUT /identity/v3/projects/449be8dc1626483c9c9f2b4a06aae479/users/0814d7e95f64451f97fa436fda771fbb/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:36:57.944840 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce777bab-1021-4680-8954-2af4d663eca2 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:57.957217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce777bab-1021-4680-8954-2af4d663eca2 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:58.041711 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c723eeb-bf53-4fe2-b75a-338ca669a912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:58.042475 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c723eeb-bf53-4fe2-b75a-338ca669a912 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:58.042926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c723eeb-bf53-4fe2-b75a-338ca669a912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:58.043119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c723eeb-bf53-4fe2-b75a-338ca669a912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:58.043886 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c723eeb-bf53-4fe2-b75a-338ca669a912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:58.046397 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c723eeb-bf53-4fe2-b75a-338ca669a912 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:58.060394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 941/1900] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:36:58.067113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d128b-8730-4ac6-a9a4-c223d3865dc8 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:58.079269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d128b-8730-4ac6-a9a4-c223d3865dc8 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:58.085266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0538369-bcda-4452-9e39-41798096bd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:58.086250 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0538369-bcda-4452-9e39-41798096bd7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:58.086605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0538369-bcda-4452-9e39-41798096bd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:58.086951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0538369-bcda-4452-9e39-41798096bd7e admin admin] PATH_INFO: `/v3/projects/449be8dc1626483c9c9f2b4a06aae479/users/0814d7e95f64451f97fa436fda771fbb/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:58.089155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0538369-bcda-4452-9e39-41798096bd7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=449be8dc1626483c9c9f2b4a06aae479, user_id=0814d7e95f64451f97fa436fda771fbb, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:58.092195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0538369-bcda-4452-9e39-41798096bd7e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:58.106286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 960/1901] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:36:58 2022] PUT /identity/v3/projects/449be8dc1626483c9c9f2b4a06aae479/users/0814d7e95f64451f97fa436fda771fbb/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:36:58.118719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32e1ad-f8d3-4222-b07b-54a589d78a06 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:58.119233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32e1ad-f8d3-4222-b07b-54a589d78a06 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:58.119511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32e1ad-f8d3-4222-b07b-54a589d78a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:58.148105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-bd32e1ad-f8d3-4222-b07b-54a589d78a06 None None] MFA Rules not processed for user `0814d7e95f64451f97fa436fda771fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:58.149350 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bd32e1ad-f8d3-4222-b07b-54a589d78a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:58.219610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 942/1902] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:36:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:58.230956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0538369-bcda-4452-9e39-41798096bd7e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:58.232676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a0538369-bcda-4452-9e39-41798096bd7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:58.266818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:58.295453 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0538369-bcda-4452-9e39-41798096bd7e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:58.299202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc27640-ad93-47d5-8cd3-5600f7e27d69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:58.299851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc27640-ad93-47d5-8cd3-5600f7e27d69 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:58.300131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc27640-ad93-47d5-8cd3-5600f7e27d69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:58.300409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc27640-ad93-47d5-8cd3-5600f7e27d69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:58.359694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc27640-ad93-47d5-8cd3-5600f7e27d69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:58.361402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc27640-ad93-47d5-8cd3-5600f7e27d69 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:58.374787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 961/1903] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:06.820701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32e1ad-f8d3-4222-b07b-54a589d78a06 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:06.823377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bd32e1ad-f8d3-4222-b07b-54a589d78a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:06.838164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32e1ad-f8d3-4222-b07b-54a589d78a06 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:06.845214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd25f5-0aae-4fb3-b6d3-f885717469f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:06.846340 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd25f5-0aae-4fb3-b6d3-f885717469f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:06.846873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd25f5-0aae-4fb3-b6d3-f885717469f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:06.847372 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd25f5-0aae-4fb3-b6d3-f885717469f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:06.893866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:06.925112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd25f5-0aae-4fb3-b6d3-f885717469f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:06.927136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd25f5-0aae-4fb3-b6d3-f885717469f3 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:06.937511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 943/1904] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:07.712521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc27640-ad93-47d5-8cd3-5600f7e27d69 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:07.722128 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc27640-ad93-47d5-8cd3-5600f7e27d69 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:07.727577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fd894-564c-4d5f-8eff-3cf18cdd19d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:07.728371 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fd894-564c-4d5f-8eff-3cf18cdd19d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:07.728899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fd894-564c-4d5f-8eff-3cf18cdd19d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:07.729529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fd894-564c-4d5f-8eff-3cf18cdd19d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:07.761589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fd894-564c-4d5f-8eff-3cf18cdd19d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:07.763500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fd894-564c-4d5f-8eff-3cf18cdd19d2 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:07.776199 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 962/1905] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:09.764058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd25f5-0aae-4fb3-b6d3-f885717469f3 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:09.766037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0dbd25f5-0aae-4fb3-b6d3-f885717469f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:09.917221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd25f5-0aae-4fb3-b6d3-f885717469f3 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:09.922124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:09.923177 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:09.923621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:09.924032 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] PATH_INFO: `/v3/users/1ede400146594b19ad3d922788ae6fe0` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:09.925452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ede400146594b19ad3d922788ae6fe0)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:09.927559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:09.994787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ede400146594b19ad3d922788ae6fe0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:09.995506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ede400146594b19ad3d922788ae6fe0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:10.019457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ede400146594b19ad3d922788ae6fe0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:10.020186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ede400146594b19ad3d922788ae6fe0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:10.046635 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ede400146594b19ad3d922788ae6fe0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:10.081369 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 944/1906] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:09 2022] DELETE /identity/v3/users/1ede400146594b19ad3d922788ae6fe0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:10.085702 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2092c4-b3d9-4249-9ccd-93dd981446e0 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:10.086618 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2092c4-b3d9-4249-9ccd-93dd981446e0 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:10.086618 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2092c4-b3d9-4249-9ccd-93dd981446e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:10.109769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-5e2092c4-b3d9-4249-9ccd-93dd981446e0 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:10.111268 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5e2092c4-b3d9-4249-9ccd-93dd981446e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:10.217889 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 963/1907] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:10.230111 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:10.231554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:10.303650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be031338-953c-47ae-8852-4eb51ee3ab10 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:10.307943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf8f1d-4218-4630-a033-1f825ffec3d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:10.308801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf8f1d-4218-4630-a033-1f825ffec3d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:10.309102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf8f1d-4218-4630-a033-1f825ffec3d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:10.309387 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf8f1d-4218-4630-a033-1f825ffec3d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:10.315798 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:10.322234 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf8f1d-4218-4630-a033-1f825ffec3d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:10.323811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf8f1d-4218-4630-a033-1f825ffec3d1 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:10.340831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 945/1908] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:11.570914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2092c4-b3d9-4249-9ccd-93dd981446e0 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:11.572382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5e2092c4-b3d9-4249-9ccd-93dd981446e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:11.598688 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2092c4-b3d9-4249-9ccd-93dd981446e0 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:11.614967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:11.616968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:11.617394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:11.617792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] PATH_INFO: `/v3/projects/122ab0b07ec1444fa7565431811ca511` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:11.620392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=122ab0b07ec1444fa7565431811ca511)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:11.627390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:11.784031 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '122ab0b07ec1444fa7565431811ca511', 'request_id': 'req-3000646c-7865-40c7-a5c1-e56410fee2f6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:11.792509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '122ab0b07ec1444fa7565431811ca511', 'request_id': 'req-3000646c-7865-40c7-a5c1-e56410fee2f6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:11.806525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '122ab0b07ec1444fa7565431811ca511', 'request_id': 'req-3000646c-7865-40c7-a5c1-e56410fee2f6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:11.810896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:11.827079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] The token cache is being invalidate because project 122ab0b07ec1444fa7565431811ca511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:11.827735 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:11.828686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:11.837512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf8f1d-4218-4630-a033-1f825ffec3d1 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:11.838978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 964/1909] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:11 2022] DELETE /identity/v3/projects/122ab0b07ec1444fa7565431811ca511 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:11.851624 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf8f1d-4218-4630-a033-1f825ffec3d1 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:11.980802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:11.982732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:11.983310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09e30a-8fd5-4ba5-b28b-e06ca13006f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:11.984304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09e30a-8fd5-4ba5-b28b-e06ca13006f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:11.984665 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09e30a-8fd5-4ba5-b28b-e06ca13006f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:11.985011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09e30a-8fd5-4ba5-b28b-e06ca13006f4 admin admin] PATH_INFO: `/v3/users/f9a36cab4e5c444f9bc81bf35ca81ead` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:11.987265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09e30a-8fd5-4ba5-b28b-e06ca13006f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9a36cab4e5c444f9bc81bf35ca81ead)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:11.989542 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09e30a-8fd5-4ba5-b28b-e06ca13006f4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:12.005469 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000646c-7865-40c7-a5c1-e56410fee2f6 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:12.020472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:12.023225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:12.023909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:12.024425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] PATH_INFO: `/v3/users/b82e8e695d5745128e6ac6ff8198b163` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:12.029315 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b82e8e695d5745128e6ac6ff8198b163)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:12.033601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:12.080891 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aa09e30a-8fd5-4ba5-b28b-e06ca13006f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9a36cab4e5c444f9bc81bf35ca81ead'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:12.082219 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aa09e30a-8fd5-4ba5-b28b-e06ca13006f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9a36cab4e5c444f9bc81bf35ca81ead'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:12.107564 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aa09e30a-8fd5-4ba5-b28b-e06ca13006f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9a36cab4e5c444f9bc81bf35ca81ead'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:12.110153 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aa09e30a-8fd5-4ba5-b28b-e06ca13006f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9a36cab4e5c444f9bc81bf35ca81ead'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:12.147726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b82e8e695d5745128e6ac6ff8198b163'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:12.149215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b82e8e695d5745128e6ac6ff8198b163'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:12.163847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-aa09e30a-8fd5-4ba5-b28b-e06ca13006f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9a36cab4e5c444f9bc81bf35ca81ead'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:12.173850 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b82e8e695d5745128e6ac6ff8198b163'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:12.206829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 946/1910] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:11 2022] DELETE /identity/v3/users/f9a36cab4e5c444f9bc81bf35ca81ead => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:12.212747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b82e8e695d5745128e6ac6ff8198b163'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:12.213533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b82e8e695d5745128e6ac6ff8198b163'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:12.234893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 965/1911] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:11 2022] DELETE /identity/v3/users/b82e8e695d5745128e6ac6ff8198b163 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:12.239994 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947aa024-2dd4-457c-bfb4-fb757c3f68d8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:12.240554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947aa024-2dd4-457c-bfb4-fb757c3f68d8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:12.241617 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947aa024-2dd4-457c-bfb4-fb757c3f68d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:12.269145 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-947aa024-2dd4-457c-bfb4-fb757c3f68d8 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:12.270966 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-947aa024-2dd4-457c-bfb4-fb757c3f68d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:12.420965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 947/1912] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:12.444436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:12.446296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:12.562822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9e309-ad65-4170-9b2b-983e3698b404 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:12.589550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2963ad9-443c-4f1c-8feb-2f99afb461db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:12.590295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2963ad9-443c-4f1c-8feb-2f99afb461db service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:12.590520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2963ad9-443c-4f1c-8feb-2f99afb461db service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:12.590776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2963ad9-443c-4f1c-8feb-2f99afb461db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:12.603851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2963ad9-443c-4f1c-8feb-2f99afb461db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:12.605352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2963ad9-443c-4f1c-8feb-2f99afb461db service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:12.632502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 966/1913] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:37:13.200468 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947aa024-2dd4-457c-bfb4-fb757c3f68d8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:13.203481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2963ad9-443c-4f1c-8feb-2f99afb461db service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:13.204413 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-947aa024-2dd4-457c-bfb4-fb757c3f68d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:13.208808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b2963ad9-443c-4f1c-8feb-2f99afb461db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:13.247727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947aa024-2dd4-457c-bfb4-fb757c3f68d8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:13.250602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2963ad9-443c-4f1c-8feb-2f99afb461db service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:13.255599 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:13.257485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:13.257988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:13.258485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f admin admin] PATH_INFO: `/v3/projects/f06245f01c7e47cca85cf49888d55270` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:13.260184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f06245f01c7e47cca85cf49888d55270)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:13.260307 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:13.261201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:13.261380 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:13.261773 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] PATH_INFO: `/v3/projects/36eadc7b786244a7a5021e55ba7c3068` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:13.263474 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36eadc7b786244a7a5021e55ba7c3068)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:13.265398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:13.265732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:13.275735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:13.332259 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36eadc7b786244a7a5021e55ba7c3068', 'request_id': 'req-014f231a-6139-4eaf-9860-dcaed089c851'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.332825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f06245f01c7e47cca85cf49888d55270', 'request_id': 'req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.340278 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36eadc7b786244a7a5021e55ba7c3068', 'request_id': 'req-014f231a-6139-4eaf-9860-dcaed089c851'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.342082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f06245f01c7e47cca85cf49888d55270', 'request_id': 'req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.350966 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36eadc7b786244a7a5021e55ba7c3068', 'request_id': 'req-014f231a-6139-4eaf-9860-dcaed089c851'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.355611 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f06245f01c7e47cca85cf49888d55270', 'request_id': 'req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.358739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:13.367945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] The token cache is being invalidate because project 36eadc7b786244a7a5021e55ba7c3068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:13.368623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.369276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.371114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 948/1914] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:13 2022] DELETE /identity/v3/projects/36eadc7b786244a7a5021e55ba7c3068 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:13.375455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:13.378339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f admin admin] The token cache is being invalidate because project f06245f01c7e47cca85cf49888d55270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:13.379358 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:13.380093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.381019 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b61771c9-3d62-4ed0-bf2a-0d00e7aeef3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.384781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 967/1915] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:13 2022] DELETE /identity/v3/projects/f06245f01c7e47cca85cf49888d55270 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:13.389678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6631f-83bf-4c1c-90df-1bd270bb0190 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:13.390135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6631f-83bf-4c1c-90df-1bd270bb0190 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:13.390418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6631f-83bf-4c1c-90df-1bd270bb0190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:13.429767 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-6db6631f-83bf-4c1c-90df-1bd270bb0190 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:13.432358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6db6631f-83bf-4c1c-90df-1bd270bb0190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:13.471726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f231a-6139-4eaf-9860-dcaed089c851 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:13.477586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1e21b-fa25-4356-aba7-396291e32b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:13.478360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1e21b-fa25-4356-aba7-396291e32b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:13.478637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1e21b-fa25-4356-aba7-396291e32b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:13.478908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1e21b-fa25-4356-aba7-396291e32b8b admin admin] PATH_INFO: `/v3/users/cfdceb75d4844ea19f8de36c12796100` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:13.480144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1e21b-fa25-4356-aba7-396291e32b8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfdceb75d4844ea19f8de36c12796100)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:13.482188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1e21b-fa25-4356-aba7-396291e32b8b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:13.516679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 968/1916] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:13.525890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55171-95b3-49c2-8e59-2abaf152678e None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:13.526206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55171-95b3-49c2-8e59-2abaf152678e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:13.526438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55171-95b3-49c2-8e59-2abaf152678e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:13.542530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2ba1e21b-fa25-4356-aba7-396291e32b8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfdceb75d4844ea19f8de36c12796100'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.543582 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2ba1e21b-fa25-4356-aba7-396291e32b8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfdceb75d4844ea19f8de36c12796100'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.545163 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-78b55171-95b3-49c2-8e59-2abaf152678e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:13.546379 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-78b55171-95b3-49c2-8e59-2abaf152678e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:13.552776 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2ba1e21b-fa25-4356-aba7-396291e32b8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfdceb75d4844ea19f8de36c12796100'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.553785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2ba1e21b-fa25-4356-aba7-396291e32b8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfdceb75d4844ea19f8de36c12796100'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.556984 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 969/1917] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:13.560878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b55171-95b3-49c2-8e59-2abaf152678e None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:13.571145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b55171-95b3-49c2-8e59-2abaf152678e None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:13.571327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2ba1e21b-fa25-4356-aba7-396291e32b8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfdceb75d4844ea19f8de36c12796100'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:13.576670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984ee2f-9ecb-4eee-ba64-1c66b6050e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:13.577390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984ee2f-9ecb-4eee-ba64-1c66b6050e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:13.577681 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984ee2f-9ecb-4eee-ba64-1c66b6050e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:13.577963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984ee2f-9ecb-4eee-ba64-1c66b6050e84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:13.578705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984ee2f-9ecb-4eee-ba64-1c66b6050e84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:37:13.579022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984ee2f-9ecb-4eee-ba64-1c66b6050e84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:13.580626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984ee2f-9ecb-4eee-ba64-1c66b6050e84 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:13.599112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 949/1918] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:13 2022] DELETE /identity/v3/users/cfdceb75d4844ea19f8de36c12796100 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:13.599911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 970/1919] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:37:13.610011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1e21b-fa25-4356-aba7-396291e32b8b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:13.642692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:13.720135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1e21b-fa25-4356-aba7-396291e32b8b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:13.728186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bf736-8343-471d-acd2-30b145c4e313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:13.729214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bf736-8343-471d-acd2-30b145c4e313 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:13.729566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bf736-8343-471d-acd2-30b145c4e313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:13.729873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bf736-8343-471d-acd2-30b145c4e313 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:13.730689 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16bf736-8343-471d-acd2-30b145c4e313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:13.732910 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16bf736-8343-471d-acd2-30b145c4e313 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:13.785844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984ee2f-9ecb-4eee-ba64-1c66b6050e84 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:13.794011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 950/1920] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:13 2022] POST /identity/v3/projects => generated 387 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:13.815961 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984ee2f-9ecb-4eee-ba64-1c66b6050e84 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:13.904723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb660187-d420-4981-a8c4-47c68c062c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:13.905490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb660187-d420-4981-a8c4-47c68c062c92 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:13.905891 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb660187-d420-4981-a8c4-47c68c062c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:13.906347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb660187-d420-4981-a8c4-47c68c062c92 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:13.907071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb660187-d420-4981-a8c4-47c68c062c92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:13.910066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb660187-d420-4981-a8c4-47c68c062c92 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:13.965936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 971/1921] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:13 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:13.967054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bf736-8343-471d-acd2-30b145c4e313 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:13.985609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bf736-8343-471d-acd2-30b145c4e313 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:14.001783 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3055ee0-3e38-4459-acd6-73e1a53e3812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:14.002479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3055ee0-3e38-4459-acd6-73e1a53e3812 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:14.002755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3055ee0-3e38-4459-acd6-73e1a53e3812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:14.003037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3055ee0-3e38-4459-acd6-73e1a53e3812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:14.003762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3055ee0-3e38-4459-acd6-73e1a53e3812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:14.008950 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3055ee0-3e38-4459-acd6-73e1a53e3812 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:14.030805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb660187-d420-4981-a8c4-47c68c062c92 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:14.033590 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-eb660187-d420-4981-a8c4-47c68c062c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:14.060303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 951/1922] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:37:14.064860 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3055ee0-3e38-4459-acd6-73e1a53e3812 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:14.065598 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb660187-d420-4981-a8c4-47c68c062c92 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:14.080009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:14.082564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:14.084164 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:14.085192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] PATH_INFO: `/v3/projects/7a1e192c2d344074b8530f27e6538c5a` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:14.087382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1e192c2d344074b8530f27e6538c5a)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:14.090700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:14.095831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3055ee0-3e38-4459-acd6-73e1a53e3812 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:14.108580 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947e424-856c-466e-8ce2-289a2906732b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:14.110232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947e424-856c-466e-8ce2-289a2906732b admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:14.110232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947e424-856c-466e-8ce2-289a2906732b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:14.110232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947e424-856c-466e-8ce2-289a2906732b admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:14.111038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947e424-856c-466e-8ce2-289a2906732b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:14.116155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947e424-856c-466e-8ce2-289a2906732b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:14.138325 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 952/1923] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:14.142602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947e424-856c-466e-8ce2-289a2906732b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:14.154538 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947e424-856c-466e-8ce2-289a2906732b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:14.172359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad9343-c9c8-48cf-81d3-864b94f7d64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:14.173607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad9343-c9c8-48cf-81d3-864b94f7d64b admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:14.173918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad9343-c9c8-48cf-81d3-864b94f7d64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:14.174222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad9343-c9c8-48cf-81d3-864b94f7d64b admin admin] PATH_INFO: `/v3/projects/5d08148a57024f419d1aba68d03b164b/users/f01e240e85544698880ac2ee7dbd6541/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:14.209606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad9343-c9c8-48cf-81d3-864b94f7d64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d08148a57024f419d1aba68d03b164b, user_id=f01e240e85544698880ac2ee7dbd6541, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:14.209606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad9343-c9c8-48cf-81d3-864b94f7d64b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:14.216184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1e192c2d344074b8530f27e6538c5a', 'request_id': 'req-068ca169-14a0-4c87-8494-3cef676b00c1'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:14.227352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1e192c2d344074b8530f27e6538c5a', 'request_id': 'req-068ca169-14a0-4c87-8494-3cef676b00c1'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:14.229961 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:14.237153 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 953/1924] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:14 2022] PUT /identity/v3/projects/5d08148a57024f419d1aba68d03b164b/users/f01e240e85544698880ac2ee7dbd6541/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:14.249146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76328ed-9255-40d1-8840-2e6e12dbe4f2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:14.249146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76328ed-9255-40d1-8840-2e6e12dbe4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:14.249431 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76328ed-9255-40d1-8840-2e6e12dbe4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:14.251056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1e192c2d344074b8530f27e6538c5a', 'request_id': 'req-068ca169-14a0-4c87-8494-3cef676b00c1'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:14.275269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] The token cache is being invalidate because project 7a1e192c2d344074b8530f27e6538c5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:14.279115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:14.280308 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:14.283264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 972/1925] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:14 2022] DELETE /identity/v3/projects/7a1e192c2d344074b8530f27e6538c5a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:14.286838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:14.289620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:14.299172 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b76328ed-9255-40d1-8840-2e6e12dbe4f2 None None] MFA Rules not processed for user `f01e240e85544698880ac2ee7dbd6541`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:14.300503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b76328ed-9255-40d1-8840-2e6e12dbe4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:14.403894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ca169-14a0-4c87-8494-3cef676b00c1 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:14.417098 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 954/1926] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:14.420963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76328ed-9255-40d1-8840-2e6e12dbe4f2 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:14.422316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b76328ed-9255-40d1-8840-2e6e12dbe4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:14.479961 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc30dc-16e4-4cca-b349-77ad5f31a74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:14.480975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc30dc-16e4-4cca-b349-77ad5f31a74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:14.481563 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc30dc-16e4-4cca-b349-77ad5f31a74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:14.481908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc30dc-16e4-4cca-b349-77ad5f31a74d admin admin] PATH_INFO: `/v3/users/0814d7e95f64451f97fa436fda771fbb` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:14.483362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc30dc-16e4-4cca-b349-77ad5f31a74d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0814d7e95f64451f97fa436fda771fbb)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:14.485741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc30dc-16e4-4cca-b349-77ad5f31a74d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:14.507774 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:14.537252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76328ed-9255-40d1-8840-2e6e12dbe4f2 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:14.546354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a19b8-cb6b-45c2-b6c1-dc9b566276c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:14.547369 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a19b8-cb6b-45c2-b6c1-dc9b566276c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:14.547975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a19b8-cb6b-45c2-b6c1-dc9b566276c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:14.548673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a19b8-cb6b-45c2-b6c1-dc9b566276c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:14.549320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-42cc30dc-16e4-4cca-b349-77ad5f31a74d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0814d7e95f64451f97fa436fda771fbb'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:14.549705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a19b8-cb6b-45c2-b6c1-dc9b566276c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:14.550447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-42cc30dc-16e4-4cca-b349-77ad5f31a74d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0814d7e95f64451f97fa436fda771fbb'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:14.552941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a19b8-cb6b-45c2-b6c1-dc9b566276c7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:14.561871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-42cc30dc-16e4-4cca-b349-77ad5f31a74d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0814d7e95f64451f97fa436fda771fbb'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:14.592065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-42cc30dc-16e4-4cca-b349-77ad5f31a74d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0814d7e95f64451f97fa436fda771fbb'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:14.592917 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-42cc30dc-16e4-4cca-b349-77ad5f31a74d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0814d7e95f64451f97fa436fda771fbb'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:14.593253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 955/1927] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:14 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:14.596092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a19b8-cb6b-45c2-b6c1-dc9b566276c7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:14.598619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-292a19b8-cb6b-45c2-b6c1-dc9b566276c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:14.616498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 973/1928] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:14 2022] DELETE /identity/v3/users/0814d7e95f64451f97fa436fda771fbb => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:14.681660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a19b8-cb6b-45c2-b6c1-dc9b566276c7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:14.783417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4e872-7e6d-42fb-b8a0-660d174ad5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:14.784109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4e872-7e6d-42fb-b8a0-660d174ad5ab admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:14.784555 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4e872-7e6d-42fb-b8a0-660d174ad5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:14.784979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4e872-7e6d-42fb-b8a0-660d174ad5ab admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:14.785913 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4e872-7e6d-42fb-b8a0-660d174ad5ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:14.789010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4e872-7e6d-42fb-b8a0-660d174ad5ab admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:14.816499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 956/1929] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:14 2022] POST /identity/v3/users => generated 299 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:14.820776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc30dc-16e4-4cca-b349-77ad5f31a74d admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:14.831591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc30dc-16e4-4cca-b349-77ad5f31a74d admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:14.837920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4c4d9-b834-4d54-a47b-ce833cb0e943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:14.838650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4c4d9-b834-4d54-a47b-ce833cb0e943 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:14.838937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4c4d9-b834-4d54-a47b-ce833cb0e943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:14.839211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4c4d9-b834-4d54-a47b-ce833cb0e943 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:14.840093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4c4d9-b834-4d54-a47b-ce833cb0e943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:14.841747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4c4d9-b834-4d54-a47b-ce833cb0e943 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:14.844797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:14.857045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 974/1930] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:37:14.862155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4e872-7e6d-42fb-b8a0-660d174ad5ab admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:14.873524 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4e872-7e6d-42fb-b8a0-660d174ad5ab admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:14.878488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021002f-ac8d-4589-be16-ef311c81d8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:14.880088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021002f-ac8d-4589-be16-ef311c81d8d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:14.880321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021002f-ac8d-4589-be16-ef311c81d8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:14.880650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021002f-ac8d-4589-be16-ef311c81d8d8 admin admin] PATH_INFO: `/v3/projects/88e92df686fb49bdb06c231cafcb8f71/users/5bdfd671e06f41d4be9d2fdb4cc5e215/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:14.900465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021002f-ac8d-4589-be16-ef311c81d8d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e92df686fb49bdb06c231cafcb8f71, user_id=5bdfd671e06f41d4be9d2fdb4cc5e215, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:14.903650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021002f-ac8d-4589-be16-ef311c81d8d8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:14.924883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 957/1931] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:14 2022] PUT /identity/v3/projects/88e92df686fb49bdb06c231cafcb8f71/users/5bdfd671e06f41d4be9d2fdb4cc5e215/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:14.930103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4c4d9-b834-4d54-a47b-ce833cb0e943 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:14.942701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4c4d9-b834-4d54-a47b-ce833cb0e943 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:14.959835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021002f-ac8d-4589-be16-ef311c81d8d8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:14.962166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f021002f-ac8d-4589-be16-ef311c81d8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:15.016335 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4aaa8-fbf9-440b-965b-46e9aae4f7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:15.017026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4aaa8-fbf9-440b-965b-46e9aae4f7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.017350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4aaa8-fbf9-440b-965b-46e9aae4f7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.018268 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4aaa8-fbf9-440b-965b-46e9aae4f7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.019492 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4aaa8-fbf9-440b-965b-46e9aae4f7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:15.021518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4aaa8-fbf9-440b-965b-46e9aae4f7b4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:15.024946 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:15.033716 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 975/1932] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:15.039455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4aaa8-fbf9-440b-965b-46e9aae4f7b4 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:15.048791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021002f-ac8d-4589-be16-ef311c81d8d8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:15.049659 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4aaa8-fbf9-440b-965b-46e9aae4f7b4 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:15.055443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c3d8a-3bc3-4127-b802-384795be5a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:15.056771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c3d8a-3bc3-4127-b802-384795be5a58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.056893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3178a03-6b3a-4d08-b8e1-181f08c285f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:15.057233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c3d8a-3bc3-4127-b802-384795be5a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.057663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c3d8a-3bc3-4127-b802-384795be5a58 admin admin] PATH_INFO: `/v3/projects/449be8dc1626483c9c9f2b4a06aae479` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.057863 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3178a03-6b3a-4d08-b8e1-181f08c285f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.058155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3178a03-6b3a-4d08-b8e1-181f08c285f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.058439 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3178a03-6b3a-4d08-b8e1-181f08c285f4 admin admin] PATH_INFO: `/v3/projects/88e92df686fb49bdb06c231cafcb8f71/users/5bdfd671e06f41d4be9d2fdb4cc5e215/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.059530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c3d8a-3bc3-4127-b802-384795be5a58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=449be8dc1626483c9c9f2b4a06aae479)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:15.060675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3178a03-6b3a-4d08-b8e1-181f08c285f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e92df686fb49bdb06c231cafcb8f71, user_id=5bdfd671e06f41d4be9d2fdb4cc5e215, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:15.062710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c3d8a-3bc3-4127-b802-384795be5a58 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:15.063433 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3178a03-6b3a-4d08-b8e1-181f08c285f4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:15.076653 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 976/1933] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:15 2022] PUT /identity/v3/projects/88e92df686fb49bdb06c231cafcb8f71/users/5bdfd671e06f41d4be9d2fdb4cc5e215/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:15.088876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be4456-4013-48e8-86b6-2cdfc70fe37f None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.089489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be4456-4013-48e8-86b6-2cdfc70fe37f None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.089901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be4456-4013-48e8-86b6-2cdfc70fe37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.120852 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-92be4456-4013-48e8-86b6-2cdfc70fe37f None None] MFA Rules not processed for user `5bdfd671e06f41d4be9d2fdb4cc5e215`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:15.122104 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-92be4456-4013-48e8-86b6-2cdfc70fe37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:15.128564 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e81c3d8a-3bc3-4127-b802-384795be5a58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '449be8dc1626483c9c9f2b4a06aae479', 'request_id': 'req-e81c3d8a-3bc3-4127-b802-384795be5a58'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:15.137242 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e81c3d8a-3bc3-4127-b802-384795be5a58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '449be8dc1626483c9c9f2b4a06aae479', 'request_id': 'req-e81c3d8a-3bc3-4127-b802-384795be5a58'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:15.147540 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e81c3d8a-3bc3-4127-b802-384795be5a58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '449be8dc1626483c9c9f2b4a06aae479', 'request_id': 'req-e81c3d8a-3bc3-4127-b802-384795be5a58'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:15.160874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e81c3d8a-3bc3-4127-b802-384795be5a58 admin admin] The token cache is being invalidate because project 449be8dc1626483c9c9f2b4a06aae479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:15.161105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e81c3d8a-3bc3-4127-b802-384795be5a58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:15.161770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e81c3d8a-3bc3-4127-b802-384795be5a58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:15.164008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 958/1934] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:14 2022] DELETE /identity/v3/projects/449be8dc1626483c9c9f2b4a06aae479 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:15.168933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b4c0d-0006-45d1-acb2-b032ac12e09f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.169255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b4c0d-0006-45d1-acb2-b032ac12e09f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.169536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b4c0d-0006-45d1-acb2-b032ac12e09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.189379 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-140b4c0d-0006-45d1-acb2-b032ac12e09f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:15.190619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-140b4c0d-0006-45d1-acb2-b032ac12e09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:15.197130 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 977/1935] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:15.209861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be4456-4013-48e8-86b6-2cdfc70fe37f None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:15.211473 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-92be4456-4013-48e8-86b6-2cdfc70fe37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:15.263535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 959/1936] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:15.265148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c49bf-ec36-45f1-8a61-fb727114942b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.265449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c49bf-ec36-45f1-8a61-fb727114942b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.265732 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c49bf-ec36-45f1-8a61-fb727114942b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.284342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f65c49bf-ec36-45f1-8a61-fb727114942b None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:15.284552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be4456-4013-48e8-86b6-2cdfc70fe37f None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:15.285643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f65c49bf-ec36-45f1-8a61-fb727114942b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:15.291561 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01974b2-197f-4143-8d49-e67a32a64614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:15.292700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01974b2-197f-4143-8d49-e67a32a64614 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.293267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01974b2-197f-4143-8d49-e67a32a64614 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.293757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01974b2-197f-4143-8d49-e67a32a64614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.294426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 960/1937] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:15.300120 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c49bf-ec36-45f1-8a61-fb727114942b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:15.309984 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01974b2-197f-4143-8d49-e67a32a64614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:15.311456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01974b2-197f-4143-8d49-e67a32a64614 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:15.311852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c49bf-ec36-45f1-8a61-fb727114942b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:15.317271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffe94c-dd72-48d4-b0d9-0ba7cc342132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:15.318162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffe94c-dd72-48d4-b0d9-0ba7cc342132 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.318448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffe94c-dd72-48d4-b0d9-0ba7cc342132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.318780 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffe94c-dd72-48d4-b0d9-0ba7cc342132 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.319523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffe94c-dd72-48d4-b0d9-0ba7cc342132 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:37:15.319895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffe94c-dd72-48d4-b0d9-0ba7cc342132 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:15.321477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffe94c-dd72-48d4-b0d9-0ba7cc342132 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:15.324171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 978/1938] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:37:15.337005 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 961/1939] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:37:15.343684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01974b2-197f-4143-8d49-e67a32a64614 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:15.360868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffe94c-dd72-48d4-b0d9-0ba7cc342132 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:15.363155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3fffe94c-dd72-48d4-b0d9-0ba7cc342132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:15.366802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01974b2-197f-4143-8d49-e67a32a64614 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:15.375368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958942c7-b7ee-4d26-af3d-617b63bc071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:15.376502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958942c7-b7ee-4d26-af3d-617b63bc071b admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.377022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958942c7-b7ee-4d26-af3d-617b63bc071b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.377580 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958942c7-b7ee-4d26-af3d-617b63bc071b admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.378879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958942c7-b7ee-4d26-af3d-617b63bc071b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:15.378985 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffe94c-dd72-48d4-b0d9-0ba7cc342132 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:15.383096 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958942c7-b7ee-4d26-af3d-617b63bc071b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:15.383925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8264944-6227-4d11-be91-5b09bb69bcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:15.384878 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8264944-6227-4d11-be91-5b09bb69bcee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.385352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8264944-6227-4d11-be91-5b09bb69bcee admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.385715 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8264944-6227-4d11-be91-5b09bb69bcee admin admin] PATH_INFO: `/v3/users/f01e240e85544698880ac2ee7dbd6541` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.386980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8264944-6227-4d11-be91-5b09bb69bcee admin admin] RBAC: Authorizing `identity:delete_user(user_id=f01e240e85544698880ac2ee7dbd6541)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:15.389060 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8264944-6227-4d11-be91-5b09bb69bcee admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:15.392817 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:15.448426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 979/1940] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:15 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:15.449698 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958942c7-b7ee-4d26-af3d-617b63bc071b admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:15.456617 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e8264944-6227-4d11-be91-5b09bb69bcee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f01e240e85544698880ac2ee7dbd6541'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:15.457364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e8264944-6227-4d11-be91-5b09bb69bcee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f01e240e85544698880ac2ee7dbd6541'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:15.461386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:15.475785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e8264944-6227-4d11-be91-5b09bb69bcee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f01e240e85544698880ac2ee7dbd6541'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:15.478096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e8264944-6227-4d11-be91-5b09bb69bcee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f01e240e85544698880ac2ee7dbd6541'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:15.509814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e8264944-6227-4d11-be91-5b09bb69bcee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f01e240e85544698880ac2ee7dbd6541'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:15.540974 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 962/1941] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:15 2022] DELETE /identity/v3/users/f01e240e85544698880ac2ee7dbd6541 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:15.548196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa66c0-6dd3-4b4c-aa97-85b9a84fec2f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.549146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa66c0-6dd3-4b4c-aa97-85b9a84fec2f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.549734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa66c0-6dd3-4b4c-aa97-85b9a84fec2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.566271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958942c7-b7ee-4d26-af3d-617b63bc071b admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:15.592999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5bfa66c0-6dd3-4b4c-aa97-85b9a84fec2f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:15.594305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5bfa66c0-6dd3-4b4c-aa97-85b9a84fec2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:15.667489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f79d5-247a-4317-a0c0-f4e0d65b3f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:15.668096 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f79d5-247a-4317-a0c0-f4e0d65b3f6c admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.668379 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f79d5-247a-4317-a0c0-f4e0d65b3f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.668686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f79d5-247a-4317-a0c0-f4e0d65b3f6c admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.669453 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f79d5-247a-4317-a0c0-f4e0d65b3f6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:15.670458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 963/1942] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:15.674259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f79d5-247a-4317-a0c0-f4e0d65b3f6c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:15.682248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa66c0-6dd3-4b4c-aa97-85b9a84fec2f None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:15.683693 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5bfa66c0-6dd3-4b4c-aa97-85b9a84fec2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:15.697502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 980/1943] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:15 2022] POST /identity/v3/users => generated 287 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:15.701877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f79d5-247a-4317-a0c0-f4e0d65b3f6c admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:15.703686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9d5f79d5-247a-4317-a0c0-f4e0d65b3f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:15.721407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f79d5-247a-4317-a0c0-f4e0d65b3f6c admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:15.727640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e0a9a-625c-40ed-bd4f-d3a075931234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:15.728428 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e0a9a-625c-40ed-bd4f-d3a075931234 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.728688 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e0a9a-625c-40ed-bd4f-d3a075931234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.729001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e0a9a-625c-40ed-bd4f-d3a075931234 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.729757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e0a9a-625c-40ed-bd4f-d3a075931234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:15.731403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e0a9a-625c-40ed-bd4f-d3a075931234 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:15.748514 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 981/1944] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:37:15.750279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e0a9a-625c-40ed-bd4f-d3a075931234 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:15.763338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e0a9a-625c-40ed-bd4f-d3a075931234 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:15.764180 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa66c0-6dd3-4b4c-aa97-85b9a84fec2f None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:15.769299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927a135-afb5-49dc-be07-31d3aace3bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:15.770077 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927a135-afb5-49dc-be07-31d3aace3bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.770340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330669d-1a69-44a4-9491-e2995f17dc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:15.770501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927a135-afb5-49dc-be07-31d3aace3bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.770820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927a135-afb5-49dc-be07-31d3aace3bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.771025 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330669d-1a69-44a4-9491-e2995f17dc13 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.771318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330669d-1a69-44a4-9491-e2995f17dc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.771780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330669d-1a69-44a4-9491-e2995f17dc13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.773401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330669d-1a69-44a4-9491-e2995f17dc13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:15.775714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330669d-1a69-44a4-9491-e2995f17dc13 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:15.790240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927a135-afb5-49dc-be07-31d3aace3bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:15.790240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927a135-afb5-49dc-be07-31d3aace3bf9 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:15.796653 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 982/1945] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:37:15.798027 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330669d-1a69-44a4-9491-e2995f17dc13 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:15.799860 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 964/1946] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:37:15.809836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330669d-1a69-44a4-9491-e2995f17dc13 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:15.815404 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39e007-9ff1-44a1-a60a-0fa8b0d84892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:15.816306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39e007-9ff1-44a1-a60a-0fa8b0d84892 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.816589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39e007-9ff1-44a1-a60a-0fa8b0d84892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.816867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39e007-9ff1-44a1-a60a-0fa8b0d84892 admin admin] PATH_INFO: `/v3/projects/2ebcc3d147b546299812829603b173d2/users/ffa54c295b764c07ba5d4c1d911fe7ce/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.839570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39e007-9ff1-44a1-a60a-0fa8b0d84892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ebcc3d147b546299812829603b173d2, user_id=ffa54c295b764c07ba5d4c1d911fe7ce, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:15.844861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39e007-9ff1-44a1-a60a-0fa8b0d84892 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:15.864483 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 983/1947] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:15 2022] PUT /identity/v3/projects/2ebcc3d147b546299812829603b173d2/users/ffa54c295b764c07ba5d4c1d911fe7ce/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:15.873852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e8ab4-bcba-437d-b5de-317bf2b0491e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:15.874353 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e8ab4-bcba-437d-b5de-317bf2b0491e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:15.874876 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e8ab4-bcba-437d-b5de-317bf2b0491e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:15.901372 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-dd1e8ab4-bcba-437d-b5de-317bf2b0491e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:15.902651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dd1e8ab4-bcba-437d-b5de-317bf2b0491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:15.993970 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 965/1948] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:16.034199 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39e007-9ff1-44a1-a60a-0fa8b0d84892 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:16.036360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6a39e007-9ff1-44a1-a60a-0fa8b0d84892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:16.091935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:16.111280 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39e007-9ff1-44a1-a60a-0fa8b0d84892 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:16.115068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3457ca3-52f1-437c-9bbb-d921cb0f054b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:16.116262 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3457ca3-52f1-437c-9bbb-d921cb0f054b service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:16.116552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3457ca3-52f1-437c-9bbb-d921cb0f054b service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:16.116831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3457ca3-52f1-437c-9bbb-d921cb0f054b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:16.126452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3457ca3-52f1-437c-9bbb-d921cb0f054b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:16.127907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3457ca3-52f1-437c-9bbb-d921cb0f054b service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:16.139678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 984/1949] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:16.168355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e8ab4-bcba-437d-b5de-317bf2b0491e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:16.171133 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dd1e8ab4-bcba-437d-b5de-317bf2b0491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:16.191477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e8ab4-bcba-437d-b5de-317bf2b0491e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:16.197523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:16.198653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:16.198905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:16.199169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] PATH_INFO: `/v3/projects/5d08148a57024f419d1aba68d03b164b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:16.200717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d08148a57024f419d1aba68d03b164b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:16.202824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:16.206360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:16.272723 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d08148a57024f419d1aba68d03b164b', 'request_id': 'req-f65d982d-2b20-4383-92b4-36efe99eb313'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:16.279942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d08148a57024f419d1aba68d03b164b', 'request_id': 'req-f65d982d-2b20-4383-92b4-36efe99eb313'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:16.291271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d08148a57024f419d1aba68d03b164b', 'request_id': 'req-f65d982d-2b20-4383-92b4-36efe99eb313'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:16.310623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] The token cache is being invalidate because project 5d08148a57024f419d1aba68d03b164b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:16.310623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:16.310623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:16.313178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 966/1950] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:16 2022] DELETE /identity/v3/projects/5d08148a57024f419d1aba68d03b164b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:16.314595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3457ca3-52f1-437c-9bbb-d921cb0f054b service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:16.327452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3457ca3-52f1-437c-9bbb-d921cb0f054b service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:16.410958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081323d4-05ff-4589-843e-03e345f9ca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:16.411440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081323d4-05ff-4589-843e-03e345f9ca79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:16.411847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081323d4-05ff-4589-843e-03e345f9ca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:16.412132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081323d4-05ff-4589-843e-03e345f9ca79 admin admin] PATH_INFO: `/v3/users/5bdfd671e06f41d4be9d2fdb4cc5e215` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:16.413873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081323d4-05ff-4589-843e-03e345f9ca79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bdfd671e06f41d4be9d2fdb4cc5e215)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:16.415885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081323d4-05ff-4589-843e-03e345f9ca79 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:16.492028 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-081323d4-05ff-4589-843e-03e345f9ca79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bdfd671e06f41d4be9d2fdb4cc5e215'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:16.493101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-081323d4-05ff-4589-843e-03e345f9ca79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bdfd671e06f41d4be9d2fdb4cc5e215'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:16.506865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-081323d4-05ff-4589-843e-03e345f9ca79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bdfd671e06f41d4be9d2fdb4cc5e215'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:16.527571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-081323d4-05ff-4589-843e-03e345f9ca79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bdfd671e06f41d4be9d2fdb4cc5e215'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:16.529124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-081323d4-05ff-4589-843e-03e345f9ca79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bdfd671e06f41d4be9d2fdb4cc5e215'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:16.547958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 985/1951] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:16 2022] DELETE /identity/v3/users/5bdfd671e06f41d4be9d2fdb4cc5e215 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:17.411940 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:17.414297 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:17.431471 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ea08b-374b-465c-b3d6-7de6afd72817 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:17.432368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ea08b-374b-465c-b3d6-7de6afd72817 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:17.433184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ea08b-374b-465c-b3d6-7de6afd72817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:17.455609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d982d-2b20-4383-92b4-36efe99eb313 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:17.502498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e89ea08b-374b-465c-b3d6-7de6afd72817 None None] MFA Rules not processed for user `ffa54c295b764c07ba5d4c1d911fe7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:17.505277 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e89ea08b-374b-465c-b3d6-7de6afd72817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:17.552535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab5f95-0292-42c7-b47e-33119f88316c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:17.554177 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab5f95-0292-42c7-b47e-33119f88316c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:17.554772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab5f95-0292-42c7-b47e-33119f88316c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:17.555170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab5f95-0292-42c7-b47e-33119f88316c admin admin] PATH_INFO: `/v3/projects/88e92df686fb49bdb06c231cafcb8f71` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:17.556670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab5f95-0292-42c7-b47e-33119f88316c admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e92df686fb49bdb06c231cafcb8f71)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:17.559116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab5f95-0292-42c7-b47e-33119f88316c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:17.596894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:17.632094 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 986/1952] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:17.640787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d5ab5f95-0292-42c7-b47e-33119f88316c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e92df686fb49bdb06c231cafcb8f71', 'request_id': 'req-d5ab5f95-0292-42c7-b47e-33119f88316c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:17.640983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ea08b-374b-465c-b3d6-7de6afd72817 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:17.645204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e89ea08b-374b-465c-b3d6-7de6afd72817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:17.653221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d5ab5f95-0292-42c7-b47e-33119f88316c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e92df686fb49bdb06c231cafcb8f71', 'request_id': 'req-d5ab5f95-0292-42c7-b47e-33119f88316c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:17.665673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d5ab5f95-0292-42c7-b47e-33119f88316c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e92df686fb49bdb06c231cafcb8f71', 'request_id': 'req-d5ab5f95-0292-42c7-b47e-33119f88316c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:17.686584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d5ab5f95-0292-42c7-b47e-33119f88316c admin admin] The token cache is being invalidate because project 88e92df686fb49bdb06c231cafcb8f71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:17.686878 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d5ab5f95-0292-42c7-b47e-33119f88316c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:17.687683 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d5ab5f95-0292-42c7-b47e-33119f88316c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:17.689381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 967/1953] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:17 2022] DELETE /identity/v3/projects/88e92df686fb49bdb06c231cafcb8f71 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:17.695578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050e013-3b45-472e-935c-b37772ce7af0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:17.695878 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050e013-3b45-472e-935c-b37772ce7af0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:17.696156 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050e013-3b45-472e-935c-b37772ce7af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:17.716654 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4050e013-3b45-472e-935c-b37772ce7af0 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:17.717813 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4050e013-3b45-472e-935c-b37772ce7af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:17.731963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ea08b-374b-465c-b3d6-7de6afd72817 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:17.742485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:17.800119 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1d2ca-b801-4786-aa0e-00d163311af5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:17.800781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1d2ca-b801-4786-aa0e-00d163311af5 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:17.801066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1d2ca-b801-4786-aa0e-00d163311af5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:17.801373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1d2ca-b801-4786-aa0e-00d163311af5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:17.802478 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 968/1954] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:17.805852 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-dac1d2ca-b801-4786-aa0e-00d163311af5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:17.810634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02331e7b-e91b-4bbb-8665-b27688f53c26 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:17.810634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02331e7b-e91b-4bbb-8665-b27688f53c26 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:17.810895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02331e7b-e91b-4bbb-8665-b27688f53c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:17.830332 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-02331e7b-e91b-4bbb-8665-b27688f53c26 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:17.831549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-02331e7b-e91b-4bbb-8665-b27688f53c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:17.844737 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 969/1955] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:17.845679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02331e7b-e91b-4bbb-8665-b27688f53c26 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:17.856342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02331e7b-e91b-4bbb-8665-b27688f53c26 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:17.861105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a44b2c-4514-436e-becc-a796c5101d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:17.861870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a44b2c-4514-436e-becc-a796c5101d9f admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:17.862161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a44b2c-4514-436e-becc-a796c5101d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:17.862436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a44b2c-4514-436e-becc-a796c5101d9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:17.863158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a44b2c-4514-436e-becc-a796c5101d9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:37:17.863480 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a44b2c-4514-436e-becc-a796c5101d9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:17.865102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a44b2c-4514-436e-becc-a796c5101d9f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:17.881375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 970/1956] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:37:17.882892 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a44b2c-4514-436e-becc-a796c5101d9f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:17.883894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1d2ca-b801-4786-aa0e-00d163311af5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:17.885762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1d2ca-b801-4786-aa0e-00d163311af5 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:17.895938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a44b2c-4514-436e-becc-a796c5101d9f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:17.899093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 987/1957] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:17.911176 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bca625-1aab-49bc-98fc-363add952de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:17.911930 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bca625-1aab-49bc-98fc-363add952de1 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:17.912160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bca625-1aab-49bc-98fc-363add952de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:17.913194 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bca625-1aab-49bc-98fc-363add952de1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:17.914089 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bca625-1aab-49bc-98fc-363add952de1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:17.916002 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bca625-1aab-49bc-98fc-363add952de1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:17.966822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 971/1958] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:17 2022] POST /identity/v3/projects => generated 387 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:17.975728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1d2ca-b801-4786-aa0e-00d163311af5 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:17.998671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1d2ca-b801-4786-aa0e-00d163311af5 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:18.095396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68a066-e84b-44b9-8bfa-ec1d5e074c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:18.096745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68a066-e84b-44b9-8bfa-ec1d5e074c5c admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:18.097218 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68a066-e84b-44b9-8bfa-ec1d5e074c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:18.097659 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68a066-e84b-44b9-8bfa-ec1d5e074c5c admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:18.098367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68a066-e84b-44b9-8bfa-ec1d5e074c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:18.101395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68a066-e84b-44b9-8bfa-ec1d5e074c5c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:18.108845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:18.135114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 988/1959] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:17 2022] POST /identity/v3/users => generated 294 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:18.140895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bca625-1aab-49bc-98fc-363add952de1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:18.154909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bca625-1aab-49bc-98fc-363add952de1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:18.181687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cd1ef-5e63-4928-8bdf-c4696c9d6138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:18.183513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cd1ef-5e63-4928-8bdf-c4696c9d6138 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:18.184009 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cd1ef-5e63-4928-8bdf-c4696c9d6138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:18.184522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cd1ef-5e63-4928-8bdf-c4696c9d6138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:18.185721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cd1ef-5e63-4928-8bdf-c4696c9d6138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:18.188232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cd1ef-5e63-4928-8bdf-c4696c9d6138 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:18.212279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 972/1960] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:18.214163 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68a066-e84b-44b9-8bfa-ec1d5e074c5c admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:18.238442 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68a066-e84b-44b9-8bfa-ec1d5e074c5c admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:18.258247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d894c8-8992-4107-b700-bddef3061e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:18.258946 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d894c8-8992-4107-b700-bddef3061e9a admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:18.259236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d894c8-8992-4107-b700-bddef3061e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:18.259523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d894c8-8992-4107-b700-bddef3061e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:18.260265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d894c8-8992-4107-b700-bddef3061e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:18.262231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d894c8-8992-4107-b700-bddef3061e9a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:18.284593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 989/1961] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:18.288144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cd1ef-5e63-4928-8bdf-c4696c9d6138 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:18.306744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cd1ef-5e63-4928-8bdf-c4696c9d6138 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:18.316797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ea3fd-38cd-4b7b-88ec-7fe2bf9d8742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:18.321321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ea3fd-38cd-4b7b-88ec-7fe2bf9d8742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:18.321856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ea3fd-38cd-4b7b-88ec-7fe2bf9d8742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:18.322026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ea3fd-38cd-4b7b-88ec-7fe2bf9d8742 admin admin] PATH_INFO: `/v3/projects/07c5b0f13cda4400b0e10711c06ef8d8/users/06fe20d9807b4cb9ae3aa4d73dd7d390/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:18.376982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ea3fd-38cd-4b7b-88ec-7fe2bf9d8742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07c5b0f13cda4400b0e10711c06ef8d8, user_id=06fe20d9807b4cb9ae3aa4d73dd7d390, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:18.384049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ea3fd-38cd-4b7b-88ec-7fe2bf9d8742 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:18.423271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 973/1962] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:18 2022] PUT /identity/v3/projects/07c5b0f13cda4400b0e10711c06ef8d8/users/06fe20d9807b4cb9ae3aa4d73dd7d390/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:18.443573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d8087-a892-47c6-b768-18b075529729 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:18.444634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d8087-a892-47c6-b768-18b075529729 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:18.445176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d8087-a892-47c6-b768-18b075529729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:18.500304 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-108d8087-a892-47c6-b768-18b075529729 None None] MFA Rules not processed for user `06fe20d9807b4cb9ae3aa4d73dd7d390`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:18.501651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-108d8087-a892-47c6-b768-18b075529729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:18.631291 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 990/1963] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:18.636235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ea3fd-38cd-4b7b-88ec-7fe2bf9d8742 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:18.766846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ea3fd-38cd-4b7b-88ec-7fe2bf9d8742 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:18.778119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b073a8-464c-4296-8576-6663f2ca57cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:18.780741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b073a8-464c-4296-8576-6663f2ca57cd admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:18.781441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b073a8-464c-4296-8576-6663f2ca57cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:18.781915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b073a8-464c-4296-8576-6663f2ca57cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:18.783120 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b073a8-464c-4296-8576-6663f2ca57cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:18.786343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b073a8-464c-4296-8576-6663f2ca57cd admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:18.843473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 974/1964] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:18 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:18.844796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d8087-a892-47c6-b768-18b075529729 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:18.862003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d8087-a892-47c6-b768-18b075529729 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:18.952543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d010ee-d512-4b92-b69b-a49cc05ac812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:18.953821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d010ee-d512-4b92-b69b-a49cc05ac812 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:18.954118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d010ee-d512-4b92-b69b-a49cc05ac812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:18.954518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d010ee-d512-4b92-b69b-a49cc05ac812 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:18.955107 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d010ee-d512-4b92-b69b-a49cc05ac812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:18.957124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d010ee-d512-4b92-b69b-a49cc05ac812 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:18.963835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b073a8-464c-4296-8576-6663f2ca57cd admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:18.965649 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f5b073a8-464c-4296-8576-6663f2ca57cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:18.982706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b073a8-464c-4296-8576-6663f2ca57cd admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:18.983002 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 991/1965] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:18 2022] POST /identity/v3/users => generated 300 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:37:18.987077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d010ee-d512-4b92-b69b-a49cc05ac812 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:18.988428 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f03858-c3cd-4f80-9a5e-ae50506903db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:18.989389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f03858-c3cd-4f80-9a5e-ae50506903db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:18.989676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f03858-c3cd-4f80-9a5e-ae50506903db admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:18.989952 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f03858-c3cd-4f80-9a5e-ae50506903db admin admin] PATH_INFO: `/v3/users/6a3bb7f368df40419b070182ea60a006` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:18.991192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f03858-c3cd-4f80-9a5e-ae50506903db admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a3bb7f368df40419b070182ea60a006)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:18.993348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f03858-c3cd-4f80-9a5e-ae50506903db admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:18.999090 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d010ee-d512-4b92-b69b-a49cc05ac812 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:19.008449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fbe9c-dfb6-42d7-b357-a31fe5a9a6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:19.010195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fbe9c-dfb6-42d7-b357-a31fe5a9a6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:19.010835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fbe9c-dfb6-42d7-b357-a31fe5a9a6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:19.011343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fbe9c-dfb6-42d7-b357-a31fe5a9a6a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:19.013661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fbe9c-dfb6-42d7-b357-a31fe5a9a6a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:19.018279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fbe9c-dfb6-42d7-b357-a31fe5a9a6a7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:19.033312 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:19.034683 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 992/1966] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:19.041131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fbe9c-dfb6-42d7-b357-a31fe5a9a6a7 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:19.065515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-16f03858-c3cd-4f80-9a5e-ae50506903db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a3bb7f368df40419b070182ea60a006'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:19.066153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fbe9c-dfb6-42d7-b357-a31fe5a9a6a7 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:19.066276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-16f03858-c3cd-4f80-9a5e-ae50506903db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a3bb7f368df40419b070182ea60a006'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:19.074449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfb7d7-f22c-4f91-80b7-cf9e236d0d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:19.075732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfb7d7-f22c-4f91-80b7-cf9e236d0d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:19.076233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfb7d7-f22c-4f91-80b7-cf9e236d0d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:19.076730 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfb7d7-f22c-4f91-80b7-cf9e236d0d1b admin admin] PATH_INFO: `/v3/projects/7e4459e117854c8785346871a2299348/users/4f608847ae8a4f84b8d177d77a2ab2a0/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:19.091149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-16f03858-c3cd-4f80-9a5e-ae50506903db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a3bb7f368df40419b070182ea60a006'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:19.091926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-16f03858-c3cd-4f80-9a5e-ae50506903db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a3bb7f368df40419b070182ea60a006'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:19.110215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfb7d7-f22c-4f91-80b7-cf9e236d0d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4459e117854c8785346871a2299348, user_id=4f608847ae8a4f84b8d177d77a2ab2a0, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:19.113684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfb7d7-f22c-4f91-80b7-cf9e236d0d1b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:19.133353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 993/1967] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:19 2022] PUT /identity/v3/projects/7e4459e117854c8785346871a2299348/users/4f608847ae8a4f84b8d177d77a2ab2a0/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:19.136219 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-16f03858-c3cd-4f80-9a5e-ae50506903db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a3bb7f368df40419b070182ea60a006'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:19.138399 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfb7d7-f22c-4f91-80b7-cf9e236d0d1b admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:19.140349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-43bfb7d7-f22c-4f91-80b7-cf9e236d0d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:19.152863 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 975/1968] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:18 2022] DELETE /identity/v3/users/6a3bb7f368df40419b070182ea60a006 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:19.159382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71594d8e-3e9a-4037-bddb-0821f1fecbcb None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:19.159686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71594d8e-3e9a-4037-bddb-0821f1fecbcb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:19.160155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71594d8e-3e9a-4037-bddb-0821f1fecbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:19.184870 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:19.195191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-71594d8e-3e9a-4037-bddb-0821f1fecbcb None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:19.196443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-71594d8e-3e9a-4037-bddb-0821f1fecbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:19.283085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfb7d7-f22c-4f91-80b7-cf9e236d0d1b admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:19.285126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 976/1969] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:19.299321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff7d99-198b-44eb-a96f-ec689702a575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:19.300504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff7d99-198b-44eb-a96f-ec689702a575 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:19.300796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff7d99-198b-44eb-a96f-ec689702a575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:19.301271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff7d99-198b-44eb-a96f-ec689702a575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:19.302519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff7d99-198b-44eb-a96f-ec689702a575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:19.305333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff7d99-198b-44eb-a96f-ec689702a575 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:19.308124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71594d8e-3e9a-4037-bddb-0821f1fecbcb None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:19.310156 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-71594d8e-3e9a-4037-bddb-0821f1fecbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:19.342113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 994/1970] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:19.343091 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff7d99-198b-44eb-a96f-ec689702a575 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:19.364559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff7d99-198b-44eb-a96f-ec689702a575 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:19.378422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc4c5f-d9f9-4928-97ba-25ef1c34f2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:19.382065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc4c5f-d9f9-4928-97ba-25ef1c34f2b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:19.382577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc4c5f-d9f9-4928-97ba-25ef1c34f2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:19.382869 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc4c5f-d9f9-4928-97ba-25ef1c34f2b0 admin admin] PATH_INFO: `/v3/projects/7e4459e117854c8785346871a2299348/users/4f608847ae8a4f84b8d177d77a2ab2a0/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:19.387312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc4c5f-d9f9-4928-97ba-25ef1c34f2b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4459e117854c8785346871a2299348, user_id=4f608847ae8a4f84b8d177d77a2ab2a0, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:19.390286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc4c5f-d9f9-4928-97ba-25ef1c34f2b0 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:19.424651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 995/1971] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:19 2022] PUT /identity/v3/projects/7e4459e117854c8785346871a2299348/users/4f608847ae8a4f84b8d177d77a2ab2a0/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:19.435710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef4348-2723-4c77-bee8-f3a6cc90dad0 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:19.436006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef4348-2723-4c77-bee8-f3a6cc90dad0 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:19.436290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef4348-2723-4c77-bee8-f3a6cc90dad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:19.454650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71594d8e-3e9a-4037-bddb-0821f1fecbcb None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:19.514271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-c1ef4348-2723-4c77-bee8-f3a6cc90dad0 None None] MFA Rules not processed for user `4f608847ae8a4f84b8d177d77a2ab2a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:19.515633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c1ef4348-2723-4c77-bee8-f3a6cc90dad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:19.525797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7ffe4-27e9-41ab-9a9c-63cbd0d7ed3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:19.526983 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7ffe4-27e9-41ab-9a9c-63cbd0d7ed3e service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:19.527209 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7ffe4-27e9-41ab-9a9c-63cbd0d7ed3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:19.527552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7ffe4-27e9-41ab-9a9c-63cbd0d7ed3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:19.548981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:19.625947 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7ffe4-27e9-41ab-9a9c-63cbd0d7ed3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:19.627411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7ffe4-27e9-41ab-9a9c-63cbd0d7ed3e service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:19.639047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 977/1972] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:19.640534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e913f-b4de-4f88-83d2-fe48c5ec53d3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:19.640888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e913f-b4de-4f88-83d2-fe48c5ec53d3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:19.641334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e913f-b4de-4f88-83d2-fe48c5ec53d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:19.659014 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 996/1973] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:19.673047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9e6e913f-b4de-4f88-83d2-fe48c5ec53d3 None None] MFA Rules not processed for user `ffa54c295b764c07ba5d4c1d911fe7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:19.673047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9e6e913f-b4de-4f88-83d2-fe48c5ec53d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:19.675037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef4348-2723-4c77-bee8-f3a6cc90dad0 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:19.683061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c1ef4348-2723-4c77-bee8-f3a6cc90dad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:19.785790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 978/1974] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:19.789563 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e913f-b4de-4f88-83d2-fe48c5ec53d3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:19.791329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9e6e913f-b4de-4f88-83d2-fe48c5ec53d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:19.794463 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef4348-2723-4c77-bee8-f3a6cc90dad0 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:19.804078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35686b3f-f944-4d98-868b-6bed71a5d4cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:19.804992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35686b3f-f944-4d98-868b-6bed71a5d4cb service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:19.805580 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35686b3f-f944-4d98-868b-6bed71a5d4cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:19.805871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35686b3f-f944-4d98-868b-6bed71a5d4cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:19.827089 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35686b3f-f944-4d98-868b-6bed71a5d4cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:19.834822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35686b3f-f944-4d98-868b-6bed71a5d4cb service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:19.843227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e913f-b4de-4f88-83d2-fe48c5ec53d3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:19.851922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:19.853117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:19.853409 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:19.853694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] PATH_INFO: `/v3/users/f76215c187a24cc3813d041f836da1e9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:19.854808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 997/1975] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:19.854938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f76215c187a24cc3813d041f836da1e9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:19.857292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:19.959538 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f76215c187a24cc3813d041f836da1e9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:19.962207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f76215c187a24cc3813d041f836da1e9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:19.974561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f76215c187a24cc3813d041f836da1e9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:19.976557 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f76215c187a24cc3813d041f836da1e9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:20.002846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f76215c187a24cc3813d041f836da1e9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:20.023453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 979/1976] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:19 2022] DELETE /identity/v3/users/f76215c187a24cc3813d041f836da1e9 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:20.398661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35686b3f-f944-4d98-868b-6bed71a5d4cb service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:20.413660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35686b3f-f944-4d98-868b-6bed71a5d4cb service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:20.465708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:20.511632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:20.513610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:20.514006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:20.514859 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] PATH_INFO: `/v3/projects/c16443dacecc446194bd04317c2779f5` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:20.516757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c16443dacecc446194bd04317c2779f5)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:20.532613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:20.633489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c16443dacecc446194bd04317c2779f5', 'request_id': 'req-bec0bbff-9ecb-4518-af89-c7550548650a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:20.645887 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c16443dacecc446194bd04317c2779f5', 'request_id': 'req-bec0bbff-9ecb-4518-af89-c7550548650a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:20.656307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c16443dacecc446194bd04317c2779f5', 'request_id': 'req-bec0bbff-9ecb-4518-af89-c7550548650a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:20.674789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] The token cache is being invalidate because project c16443dacecc446194bd04317c2779f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:20.675196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:20.676262 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:20.679100 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 998/1977] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:20 2022] DELETE /identity/v3/projects/c16443dacecc446194bd04317c2779f5 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:20.683469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:20.685238 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:20.706739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b188b-c4bd-42f9-b033-322a49da0247 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:20.783857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:20.837244 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:20.838928 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:20.864130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:20.866445 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:20.867287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:20.867757 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] PATH_INFO: `/v3/users/ac52e3d679614ce59e368229517f8660` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:20.869286 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac52e3d679614ce59e368229517f8660)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:20.872164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:21.004365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac52e3d679614ce59e368229517f8660'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.004365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac52e3d679614ce59e368229517f8660'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.013528 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac52e3d679614ce59e368229517f8660'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.015578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac52e3d679614ce59e368229517f8660'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.016810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0bbff-9ecb-4518-af89-c7550548650a admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:21.022771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d16de-b51d-486f-9bc0-90762b8d74a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:21.025533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d16de-b51d-486f-9bc0-90762b8d74a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:21.025977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d16de-b51d-486f-9bc0-90762b8d74a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:21.026529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d16de-b51d-486f-9bc0-90762b8d74a4 admin admin] PATH_INFO: `/v3/users/06fe20d9807b4cb9ae3aa4d73dd7d390` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:21.027981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d16de-b51d-486f-9bc0-90762b8d74a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06fe20d9807b4cb9ae3aa4d73dd7d390)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:21.031071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d16de-b51d-486f-9bc0-90762b8d74a4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:21.033755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac52e3d679614ce59e368229517f8660'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.056111 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 980/1978] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:20 2022] DELETE /identity/v3/users/ac52e3d679614ce59e368229517f8660 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:21.057175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:21.070996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:21.110389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c71d16de-b51d-486f-9bc0-90762b8d74a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06fe20d9807b4cb9ae3aa4d73dd7d390'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.111856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c71d16de-b51d-486f-9bc0-90762b8d74a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06fe20d9807b4cb9ae3aa4d73dd7d390'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.131451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c71d16de-b51d-486f-9bc0-90762b8d74a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06fe20d9807b4cb9ae3aa4d73dd7d390'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.145015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:21.155527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c71d16de-b51d-486f-9bc0-90762b8d74a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06fe20d9807b4cb9ae3aa4d73dd7d390'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.157265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c71d16de-b51d-486f-9bc0-90762b8d74a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06fe20d9807b4cb9ae3aa4d73dd7d390'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.164547 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048673b-ea1d-4c25-b2a9-65bfaa511f90 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:21.175552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:21.176856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:21.177356 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:21.181605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 999/1979] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:20 2022] DELETE /identity/v3/users/06fe20d9807b4cb9ae3aa4d73dd7d390 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:21.186957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b6cd-c264-44cc-80fc-63d8e888b405 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:21.187564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b6cd-c264-44cc-80fc-63d8e888b405 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:21.188999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b6cd-c264-44cc-80fc-63d8e888b405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:21.194755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] PATH_INFO: `/v3/projects/21545f05eda441d1b440788e5fbfc880` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:21.200515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] RBAC: Authorizing `identity:delete_project(project_id=21545f05eda441d1b440788e5fbfc880)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:21.204053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:21.213590 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-8cb1b6cd-c264-44cc-80fc-63d8e888b405 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:21.215692 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8cb1b6cd-c264-44cc-80fc-63d8e888b405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:21.310745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1000/1980] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:21.341654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b6cd-c264-44cc-80fc-63d8e888b405 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:21.343254 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8cb1b6cd-c264-44cc-80fc-63d8e888b405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:21.367838 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:21.370235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21545f05eda441d1b440788e5fbfc880', 'request_id': 'req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.383503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21545f05eda441d1b440788e5fbfc880', 'request_id': 'req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.396420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21545f05eda441d1b440788e5fbfc880', 'request_id': 'req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.425436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] The token cache is being invalidate because project 21545f05eda441d1b440788e5fbfc880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:21.425862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.426678 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.429645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 981/1981] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:21 2022] DELETE /identity/v3/projects/21545f05eda441d1b440788e5fbfc880 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:21.433970 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:21.436073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:21.461960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b6cd-c264-44cc-80fc-63d8e888b405 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:21.535001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d0618-4a2b-45e2-88fa-277d9ca6f62a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:21.543145 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7645e7-d4eb-4bc3-a7bf-95e1d743c4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:21.544092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7645e7-d4eb-4bc3-a7bf-95e1d743c4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:21.544402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7645e7-d4eb-4bc3-a7bf-95e1d743c4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:21.544751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7645e7-d4eb-4bc3-a7bf-95e1d743c4f2 admin admin] PATH_INFO: `/v3/users/3205adc3df4246ad8f2eedc1df218ede` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:21.546073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a730b-033a-4fa9-ae54-f897a53a2058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:21.546537 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7645e7-d4eb-4bc3-a7bf-95e1d743c4f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3205adc3df4246ad8f2eedc1df218ede)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:21.546894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a730b-033a-4fa9-ae54-f897a53a2058 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:21.547245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a730b-033a-4fa9-ae54-f897a53a2058 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:21.547609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a730b-033a-4fa9-ae54-f897a53a2058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:21.549317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7645e7-d4eb-4bc3-a7bf-95e1d743c4f2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:21.549847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-827a730b-033a-4fa9-ae54-f897a53a2058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:21.567380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a730b-033a-4fa9-ae54-f897a53a2058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:21.569731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a730b-033a-4fa9-ae54-f897a53a2058 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:21.582103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1001/1982] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:21.583613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a730b-033a-4fa9-ae54-f897a53a2058 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:21.586554 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-827a730b-033a-4fa9-ae54-f897a53a2058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:21.610540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a730b-033a-4fa9-ae54-f897a53a2058 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:21.623553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0836e-2547-4cc9-b226-cdd6ee93654b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:21.625432 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0836e-2547-4cc9-b226-cdd6ee93654b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:21.625735 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0836e-2547-4cc9-b226-cdd6ee93654b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:21.626048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0836e-2547-4cc9-b226-cdd6ee93654b admin admin] PATH_INFO: `/v3/projects/303345600487488c8856a0892ef1583a` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:21.630192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0836e-2547-4cc9-b226-cdd6ee93654b admin admin] RBAC: Authorizing `identity:delete_project(project_id=303345600487488c8856a0892ef1583a)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:21.634753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0836e-2547-4cc9-b226-cdd6ee93654b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:21.638089 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1d7645e7-d4eb-4bc3-a7bf-95e1d743c4f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3205adc3df4246ad8f2eedc1df218ede'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.639187 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1d7645e7-d4eb-4bc3-a7bf-95e1d743c4f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3205adc3df4246ad8f2eedc1df218ede'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.657220 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1d7645e7-d4eb-4bc3-a7bf-95e1d743c4f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3205adc3df4246ad8f2eedc1df218ede'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.657723 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1d7645e7-d4eb-4bc3-a7bf-95e1d743c4f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3205adc3df4246ad8f2eedc1df218ede'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.700019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1d7645e7-d4eb-4bc3-a7bf-95e1d743c4f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3205adc3df4246ad8f2eedc1df218ede'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.722884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 982/1983] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:21 2022] DELETE /identity/v3/users/3205adc3df4246ad8f2eedc1df218ede => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:21.740184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:21.769006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-caa0836e-2547-4cc9-b226-cdd6ee93654b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '303345600487488c8856a0892ef1583a', 'request_id': 'req-caa0836e-2547-4cc9-b226-cdd6ee93654b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.777371 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-caa0836e-2547-4cc9-b226-cdd6ee93654b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '303345600487488c8856a0892ef1583a', 'request_id': 'req-caa0836e-2547-4cc9-b226-cdd6ee93654b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.789512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-caa0836e-2547-4cc9-b226-cdd6ee93654b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '303345600487488c8856a0892ef1583a', 'request_id': 'req-caa0836e-2547-4cc9-b226-cdd6ee93654b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.826532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-caa0836e-2547-4cc9-b226-cdd6ee93654b admin admin] The token cache is being invalidate because project 303345600487488c8856a0892ef1583a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:21.827022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-caa0836e-2547-4cc9-b226-cdd6ee93654b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.828182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-caa0836e-2547-4cc9-b226-cdd6ee93654b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:21.831629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1002/1984] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:21 2022] DELETE /identity/v3/projects/303345600487488c8856a0892ef1583a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:21.837306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fe0f8-81ab-4343-8c71-1dbdbc6dd46a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:21.837549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fe0f8-81ab-4343-8c71-1dbdbc6dd46a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:21.837824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fe0f8-81ab-4343-8c71-1dbdbc6dd46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:21.861286 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f31fe0f8-81ab-4343-8c71-1dbdbc6dd46a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:21.862752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f31fe0f8-81ab-4343-8c71-1dbdbc6dd46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:21.929203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:21.959309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5f97d-3955-43b7-9dc6-31d9fc59c281 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:21.959611 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5f97d-3955-43b7-9dc6-31d9fc59c281 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:21.959888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5f97d-3955-43b7-9dc6-31d9fc59c281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:21.971334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 983/1985] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:22.002793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e8e5f97d-3955-43b7-9dc6-31d9fc59c281 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:22.005120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e8e5f97d-3955-43b7-9dc6-31d9fc59c281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:22.028097 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1003/1986] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:22.032940 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31fe0f8-81ab-4343-8c71-1dbdbc6dd46a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:22.049035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31fe0f8-81ab-4343-8c71-1dbdbc6dd46a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:22.057343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82621d5-54c1-4c6c-88a0-f648690917fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:22.058035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82621d5-54c1-4c6c-88a0-f648690917fc admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:22.058317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82621d5-54c1-4c6c-88a0-f648690917fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:22.058592 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82621d5-54c1-4c6c-88a0-f648690917fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:22.059342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82621d5-54c1-4c6c-88a0-f648690917fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:37:22.059651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82621d5-54c1-4c6c-88a0-f648690917fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:22.062620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82621d5-54c1-4c6c-88a0-f648690917fc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:22.090613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 984/1987] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:37:22.094868 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5f97d-3955-43b7-9dc6-31d9fc59c281 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:22.105818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5f97d-3955-43b7-9dc6-31d9fc59c281 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:22.112165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01bef4-5086-41a1-9f7b-6d8a5ea4c19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:22.112945 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01bef4-5086-41a1-9f7b-6d8a5ea4c19e admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:22.113358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01bef4-5086-41a1-9f7b-6d8a5ea4c19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:22.113843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01bef4-5086-41a1-9f7b-6d8a5ea4c19e admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:22.114848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01bef4-5086-41a1-9f7b-6d8a5ea4c19e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:22.118593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01bef4-5086-41a1-9f7b-6d8a5ea4c19e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:22.149781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1004/1988] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:22 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:22.154137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82621d5-54c1-4c6c-88a0-f648690917fc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:22.167481 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82621d5-54c1-4c6c-88a0-f648690917fc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:22.255559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01bef4-5086-41a1-9f7b-6d8a5ea4c19e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:22.257968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-2b01bef4-5086-41a1-9f7b-6d8a5ea4c19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:22.272945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8158f1-5986-4b3e-9009-c05bf0702e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:22.273806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8158f1-5986-4b3e-9009-c05bf0702e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:22.274160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8158f1-5986-4b3e-9009-c05bf0702e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:22.274530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8158f1-5986-4b3e-9009-c05bf0702e80 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:22.275783 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8158f1-5986-4b3e-9009-c05bf0702e80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:22.278852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8158f1-5986-4b3e-9009-c05bf0702e80 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:22.314048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 985/1989] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:22 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:37:22.318021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8158f1-5986-4b3e-9009-c05bf0702e80 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:22.337698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8158f1-5986-4b3e-9009-c05bf0702e80 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:22.346277 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce30c6-b8f6-4fcd-afdb-33a6da7b2757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:22.346987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce30c6-b8f6-4fcd-afdb-33a6da7b2757 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:22.347273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce30c6-b8f6-4fcd-afdb-33a6da7b2757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:22.347618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce30c6-b8f6-4fcd-afdb-33a6da7b2757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:22.348709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce30c6-b8f6-4fcd-afdb-33a6da7b2757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:22.351299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce30c6-b8f6-4fcd-afdb-33a6da7b2757 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:22.368880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 986/1990] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:22.370098 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce30c6-b8f6-4fcd-afdb-33a6da7b2757 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:22.372229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-68ce30c6-b8f6-4fcd-afdb-33a6da7b2757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:22.387486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01bef4-5086-41a1-9f7b-6d8a5ea4c19e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:22.394490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:22.394825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce30c6-b8f6-4fcd-afdb-33a6da7b2757 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:22.396074 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:22.397178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:22.397906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] PATH_INFO: `/v3/projects/07c5b0f13cda4400b0e10711c06ef8d8` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:22.399498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=07c5b0f13cda4400b0e10711c06ef8d8)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:22.400772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:22.401812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:22.402256 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:22.402364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:22.402664 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] PATH_INFO: `/v3/projects/9dd8ab6ef5fb4b81a37174d88a24de36` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:22.403917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dd8ab6ef5fb4b81a37174d88a24de36)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:22.406216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:22.489855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dd8ab6ef5fb4b81a37174d88a24de36', 'request_id': 'req-b6dba7c2-85f8-4693-ac1c-baee99f6c404'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.496110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07c5b0f13cda4400b0e10711c06ef8d8', 'request_id': 'req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.497421 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dd8ab6ef5fb4b81a37174d88a24de36', 'request_id': 'req-b6dba7c2-85f8-4693-ac1c-baee99f6c404'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.503666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07c5b0f13cda4400b0e10711c06ef8d8', 'request_id': 'req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.509134 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dd8ab6ef5fb4b81a37174d88a24de36', 'request_id': 'req-b6dba7c2-85f8-4693-ac1c-baee99f6c404'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.514135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07c5b0f13cda4400b0e10711c06ef8d8', 'request_id': 'req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.528542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] The token cache is being invalidate because project 07c5b0f13cda4400b0e10711c06ef8d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:22.528920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.530391 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.532477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1005/1991] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:22 2022] DELETE /identity/v3/projects/07c5b0f13cda4400b0e10711c06ef8d8 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:22.536212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:22.538665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:22.539058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] The token cache is being invalidate because project 9dd8ab6ef5fb4b81a37174d88a24de36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:22.539596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.540471 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.542997 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 987/1992] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:22 2022] DELETE /identity/v3/projects/9dd8ab6ef5fb4b81a37174d88a24de36 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:22.543988 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:22.546117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:22.587496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:22.617826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:22.633309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89df89-0a7f-4cef-8c3f-bed5be9203bd admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:22.639184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab10c44-6098-4994-a9f9-64eb2ab532a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:22.640255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab10c44-6098-4994-a9f9-64eb2ab532a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:22.640255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab10c44-6098-4994-a9f9-64eb2ab532a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:22.640481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab10c44-6098-4994-a9f9-64eb2ab532a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:22.641241 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab10c44-6098-4994-a9f9-64eb2ab532a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:22.643068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab10c44-6098-4994-a9f9-64eb2ab532a6 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:22.644823 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dba7c2-85f8-4693-ac1c-baee99f6c404 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:22.652336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982958fc-9c6b-4925-9df6-5a4ba4f8e4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:22.653759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982958fc-9c6b-4925-9df6-5a4ba4f8e4b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:22.654243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982958fc-9c6b-4925-9df6-5a4ba4f8e4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:22.654631 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982958fc-9c6b-4925-9df6-5a4ba4f8e4b8 admin admin] PATH_INFO: `/v3/users/4f608847ae8a4f84b8d177d77a2ab2a0` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:22.657224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982958fc-9c6b-4925-9df6-5a4ba4f8e4b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f608847ae8a4f84b8d177d77a2ab2a0)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:22.659762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1006/1993] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:22.661705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade96ab-a2bb-4f5c-9257-774379a74221 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:22.662035 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade96ab-a2bb-4f5c-9257-774379a74221 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:22.662515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade96ab-a2bb-4f5c-9257-774379a74221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:22.663249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982958fc-9c6b-4925-9df6-5a4ba4f8e4b8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:22.688436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1ade96ab-a2bb-4f5c-9257-774379a74221 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:22.690551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1ade96ab-a2bb-4f5c-9257-774379a74221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:22.700256 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1007/1994] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:22.701234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade96ab-a2bb-4f5c-9257-774379a74221 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:22.702602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1ade96ab-a2bb-4f5c-9257-774379a74221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:22.719975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade96ab-a2bb-4f5c-9257-774379a74221 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:22.727919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11500c1e-a0a8-4c45-90fa-0f97a19d85b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:22.728899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11500c1e-a0a8-4c45-90fa-0f97a19d85b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:22.729781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11500c1e-a0a8-4c45-90fa-0f97a19d85b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:22.730041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11500c1e-a0a8-4c45-90fa-0f97a19d85b6 admin admin] PATH_INFO: `/v3/projects/1547556b58444fdebb8ec8641e424509/users/0b7cde3d137a44d1bb2ce419f973f61a/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:22.752636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-982958fc-9c6b-4925-9df6-5a4ba4f8e4b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f608847ae8a4f84b8d177d77a2ab2a0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.754019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-982958fc-9c6b-4925-9df6-5a4ba4f8e4b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f608847ae8a4f84b8d177d77a2ab2a0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.755112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11500c1e-a0a8-4c45-90fa-0f97a19d85b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1547556b58444fdebb8ec8641e424509, user_id=0b7cde3d137a44d1bb2ce419f973f61a, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:22.759363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11500c1e-a0a8-4c45-90fa-0f97a19d85b6 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:22.764423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-982958fc-9c6b-4925-9df6-5a4ba4f8e4b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f608847ae8a4f84b8d177d77a2ab2a0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.765282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-982958fc-9c6b-4925-9df6-5a4ba4f8e4b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f608847ae8a4f84b8d177d77a2ab2a0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.777163 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1008/1995] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:22 2022] PUT /identity/v3/projects/1547556b58444fdebb8ec8641e424509/users/0b7cde3d137a44d1bb2ce419f973f61a/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:22.778704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bc734-28fc-4904-b745-39b40e23c3f7 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:22.779069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bc734-28fc-4904-b745-39b40e23c3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:22.779416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bc734-28fc-4904-b745-39b40e23c3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:22.781869 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-982958fc-9c6b-4925-9df6-5a4ba4f8e4b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f608847ae8a4f84b8d177d77a2ab2a0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:22.801573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-ec5bc734-28fc-4904-b745-39b40e23c3f7 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:22.804074 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ec5bc734-28fc-4904-b745-39b40e23c3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:22.807829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 988/1996] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:22 2022] DELETE /identity/v3/users/4f608847ae8a4f84b8d177d77a2ab2a0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:22.812693 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a2220-f653-483f-8323-8b1a3280af73 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:22.813236 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a2220-f653-483f-8323-8b1a3280af73 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:22.813865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a2220-f653-483f-8323-8b1a3280af73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:22.864613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ca5a2220-f653-483f-8323-8b1a3280af73 None None] MFA Rules not processed for user `0b7cde3d137a44d1bb2ce419f973f61a`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:22.868091 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ca5a2220-f653-483f-8323-8b1a3280af73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:22.927601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1009/1997] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:22.929286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bc734-28fc-4904-b745-39b40e23c3f7 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:22.939222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bc734-28fc-4904-b745-39b40e23c3f7 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:23.058757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41bf00-b6f4-43fb-abaf-877981f78954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:23.059706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41bf00-b6f4-43fb-abaf-877981f78954 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:23.060173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41bf00-b6f4-43fb-abaf-877981f78954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:23.060818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41bf00-b6f4-43fb-abaf-877981f78954 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:23.061979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41bf00-b6f4-43fb-abaf-877981f78954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:37:23.062488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41bf00-b6f4-43fb-abaf-877981f78954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:23.064422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41bf00-b6f4-43fb-abaf-877981f78954 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:23.064925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 989/1998] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:23.081214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1010/1999] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:37:23.082483 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a2220-f653-483f-8323-8b1a3280af73 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:23.085643 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ca5a2220-f653-483f-8323-8b1a3280af73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:23.087678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41bf00-b6f4-43fb-abaf-877981f78954 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:23.125953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41bf00-b6f4-43fb-abaf-877981f78954 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:23.132220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a9bf9-2a5a-4544-80c6-7a7eaeb080d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:23.132962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a9bf9-2a5a-4544-80c6-7a7eaeb080d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:23.133342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a9bf9-2a5a-4544-80c6-7a7eaeb080d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:23.133772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a9bf9-2a5a-4544-80c6-7a7eaeb080d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:23.134701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a9bf9-2a5a-4544-80c6-7a7eaeb080d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:23.137476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a9bf9-2a5a-4544-80c6-7a7eaeb080d3 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:23.186431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1011/2000] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:23 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:23.193200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a9bf9-2a5a-4544-80c6-7a7eaeb080d3 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:23.200869 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:23.264851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a2220-f653-483f-8323-8b1a3280af73 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:23.270856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:23.343249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a9bf9-2a5a-4544-80c6-7a7eaeb080d3 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:23.369006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349c05f-10af-47ec-a79d-a872f205583a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:23.370005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349c05f-10af-47ec-a79d-a872f205583a admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:23.370224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349c05f-10af-47ec-a79d-a872f205583a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:23.371116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349c05f-10af-47ec-a79d-a872f205583a admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:23.374497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349c05f-10af-47ec-a79d-a872f205583a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:23.379737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349c05f-10af-47ec-a79d-a872f205583a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:23.382606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460cded4-cbfc-4020-8d4e-59db42f511d5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e4531ac696445ba5bfa9870984befd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:23.383777 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cded4-cbfc-4020-8d4e-59db42f511d5 service glance] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:23.384391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cded4-cbfc-4020-8d4e-59db42f511d5 service glance] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:23.384885 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cded4-cbfc-4020-8d4e-59db42f511d5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:23.425292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1012/2001] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:23 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:23.425292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349c05f-10af-47ec-a79d-a872f205583a admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:23.428809 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d349c05f-10af-47ec-a79d-a872f205583a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:23.469157 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349c05f-10af-47ec-a79d-a872f205583a admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:23.476216 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72378dd-99cd-4837-9594-be2ada5329cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:23.477223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72378dd-99cd-4837-9594-be2ada5329cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:23.477509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72378dd-99cd-4837-9594-be2ada5329cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:23.477790 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72378dd-99cd-4837-9594-be2ada5329cb admin admin] PATH_INFO: `/v3/projects/7e4459e117854c8785346871a2299348` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:23.479105 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72378dd-99cd-4837-9594-be2ada5329cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e4459e117854c8785346871a2299348)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:23.481775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72378dd-99cd-4837-9594-be2ada5329cb admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:23.506034 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460cded4-cbfc-4020-8d4e-59db42f511d5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:23.508862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460cded4-cbfc-4020-8d4e-59db42f511d5 service glance] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:23.523031 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 990/2002] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:23.524429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460cded4-cbfc-4020-8d4e-59db42f511d5 service glance] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:23.549937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460cded4-cbfc-4020-8d4e-59db42f511d5 service glance] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:23.551083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a72378dd-99cd-4837-9594-be2ada5329cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e4459e117854c8785346871a2299348', 'request_id': 'req-a72378dd-99cd-4837-9594-be2ada5329cb'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:23.559235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a72378dd-99cd-4837-9594-be2ada5329cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e4459e117854c8785346871a2299348', 'request_id': 'req-a72378dd-99cd-4837-9594-be2ada5329cb'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:23.569996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a72378dd-99cd-4837-9594-be2ada5329cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e4459e117854c8785346871a2299348', 'request_id': 'req-a72378dd-99cd-4837-9594-be2ada5329cb'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:23.583379 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a72378dd-99cd-4837-9594-be2ada5329cb admin admin] The token cache is being invalidate because project 7e4459e117854c8785346871a2299348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:23.583662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a72378dd-99cd-4837-9594-be2ada5329cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:23.584532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a72378dd-99cd-4837-9594-be2ada5329cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:23.591339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1013/2003] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:23 2022] DELETE /identity/v3/projects/7e4459e117854c8785346871a2299348 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:23.593786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30658c-a697-4038-9030-bba40d3db485 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:23.594121 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30658c-a697-4038-9030-bba40d3db485 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:23.594738 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30658c-a697-4038-9030-bba40d3db485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:23.622244 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-2e30658c-a697-4038-9030-bba40d3db485 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:23.623611 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-2e30658c-a697-4038-9030-bba40d3db485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:23.700433 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1014/2004] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:23.714013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883e059-3a5a-4d54-8d24-4ee634b71edd None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:23.714318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883e059-3a5a-4d54-8d24-4ee634b71edd None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:23.714604 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883e059-3a5a-4d54-8d24-4ee634b71edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:23.736939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-f883e059-3a5a-4d54-8d24-4ee634b71edd None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:23.738941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f883e059-3a5a-4d54-8d24-4ee634b71edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:23.750468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1015/2005] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:23.751439 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883e059-3a5a-4d54-8d24-4ee634b71edd None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:23.789725 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883e059-3a5a-4d54-8d24-4ee634b71edd None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:23.814251 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8ab5d-dd10-4527-92ce-4b07e8113f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:23.815010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8ab5d-dd10-4527-92ce-4b07e8113f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:23.817563 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8ab5d-dd10-4527-92ce-4b07e8113f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:23.817902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8ab5d-dd10-4527-92ce-4b07e8113f30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:23.820105 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8ab5d-dd10-4527-92ce-4b07e8113f30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:37:23.820425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8ab5d-dd10-4527-92ce-4b07e8113f30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:23.827441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8ab5d-dd10-4527-92ce-4b07e8113f30 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:23.842791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:23.865118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1016/2006] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:37:23.866117 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8ab5d-dd10-4527-92ce-4b07e8113f30 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:23.876576 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8ab5d-dd10-4527-92ce-4b07e8113f30 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:23.882499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674356a-7ca5-40cc-9385-b143ceba821e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:23.883172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674356a-7ca5-40cc-9385-b143ceba821e admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:23.883458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674356a-7ca5-40cc-9385-b143ceba821e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:23.883734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674356a-7ca5-40cc-9385-b143ceba821e admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:23.884436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674356a-7ca5-40cc-9385-b143ceba821e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:23.886515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674356a-7ca5-40cc-9385-b143ceba821e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:23.923862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c788ca-38c0-4300-a1c6-0f3332d84147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:23.925251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c788ca-38c0-4300-a1c6-0f3332d84147 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:23.925545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c788ca-38c0-4300-a1c6-0f3332d84147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:23.925858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c788ca-38c0-4300-a1c6-0f3332d84147 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:23.927436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c788ca-38c0-4300-a1c6-0f3332d84147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:23.930530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1017/2007] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:23 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:23.932332 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c788ca-38c0-4300-a1c6-0f3332d84147 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:23.935043 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674356a-7ca5-40cc-9385-b143ceba821e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:23.948163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 991/2008] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:23.952756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c788ca-38c0-4300-a1c6-0f3332d84147 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:23.954573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c0c788ca-38c0-4300-a1c6-0f3332d84147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:23.971459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c788ca-38c0-4300-a1c6-0f3332d84147 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:24.093758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674356a-7ca5-40cc-9385-b143ceba821e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:24.110262 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad82bc5-673a-4fcd-a6b2-7b386ab6ef7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:24.113069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad82bc5-673a-4fcd-a6b2-7b386ab6ef7c admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:24.116983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad82bc5-673a-4fcd-a6b2-7b386ab6ef7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:24.117555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad82bc5-673a-4fcd-a6b2-7b386ab6ef7c admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:24.119081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad82bc5-673a-4fcd-a6b2-7b386ab6ef7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:24.128910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad82bc5-673a-4fcd-a6b2-7b386ab6ef7c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:24.148051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed280432-55ac-4442-b982-16ca60021b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:24.148831 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed280432-55ac-4442-b982-16ca60021b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:24.149207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed280432-55ac-4442-b982-16ca60021b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:24.149598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed280432-55ac-4442-b982-16ca60021b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:24.153385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed280432-55ac-4442-b982-16ca60021b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:24.155099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed280432-55ac-4442-b982-16ca60021b66 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:24.159241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:24.167333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1018/2009] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:23 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:24.168801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad82bc5-673a-4fcd-a6b2-7b386ab6ef7c admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:24.181813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad82bc5-673a-4fcd-a6b2-7b386ab6ef7c admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:24.183671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 992/2010] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:24.187868 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638ceac-63f2-4f67-8780-9c2a769f62a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:24.188780 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed280432-55ac-4442-b982-16ca60021b66 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:24.189532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638ceac-63f2-4f67-8780-9c2a769f62a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:24.190176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638ceac-63f2-4f67-8780-9c2a769f62a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:24.190607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638ceac-63f2-4f67-8780-9c2a769f62a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:24.191518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638ceac-63f2-4f67-8780-9c2a769f62a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:24.194702 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638ceac-63f2-4f67-8780-9c2a769f62a7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:24.212927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1019/2011] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:24.217178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed280432-55ac-4442-b982-16ca60021b66 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:24.217396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638ceac-63f2-4f67-8780-9c2a769f62a7 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:24.224070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db559c-05a9-4c93-8bfb-3cf748f68ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:24.225191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db559c-05a9-4c93-8bfb-3cf748f68ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:24.225550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db559c-05a9-4c93-8bfb-3cf748f68ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:24.225905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db559c-05a9-4c93-8bfb-3cf748f68ac3 admin admin] PATH_INFO: `/v3/projects/a6394338ffb2436ab6237cfb2f186e8b/users/a1b1f7ccbb09402eb057f2c6e5af7ed1/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:24.233861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638ceac-63f2-4f67-8780-9c2a769f62a7 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:24.241501 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd0ad6-2e21-4d16-a82a-77b7ef316b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:24.242956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd0ad6-2e21-4d16-a82a-77b7ef316b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:24.243406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd0ad6-2e21-4d16-a82a-77b7ef316b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:24.243838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd0ad6-2e21-4d16-a82a-77b7ef316b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:24.244739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd0ad6-2e21-4d16-a82a-77b7ef316b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:24.246694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd0ad6-2e21-4d16-a82a-77b7ef316b01 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:24.260446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db559c-05a9-4c93-8bfb-3cf748f68ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6394338ffb2436ab6237cfb2f186e8b, user_id=a1b1f7ccbb09402eb057f2c6e5af7ed1, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:24.263132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1020/2012] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:24.266583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db559c-05a9-4c93-8bfb-3cf748f68ac3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:24.267904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd0ad6-2e21-4d16-a82a-77b7ef316b01 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:24.280669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd0ad6-2e21-4d16-a82a-77b7ef316b01 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:24.284044 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 993/2013] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:24 2022] PUT /identity/v3/projects/a6394338ffb2436ab6237cfb2f186e8b/users/a1b1f7ccbb09402eb057f2c6e5af7ed1/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:24.298006 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76968b54-d07f-4ca5-9c61-50530bf1e123 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:24.298698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76968b54-d07f-4ca5-9c61-50530bf1e123 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:24.299249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76968b54-d07f-4ca5-9c61-50530bf1e123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:24.374952 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-76968b54-d07f-4ca5-9c61-50530bf1e123 None None] MFA Rules not processed for user `a1b1f7ccbb09402eb057f2c6e5af7ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:24.379608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-76968b54-d07f-4ca5-9c61-50530bf1e123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:24.402337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caa92c-208c-4bf7-a408-47426b1debe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:24.405660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caa92c-208c-4bf7-a408-47426b1debe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:24.405967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caa92c-208c-4bf7-a408-47426b1debe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:24.406257 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caa92c-208c-4bf7-a408-47426b1debe2 admin admin] PATH_INFO: `/v3/projects/d918fdf57ad64d1297959c1153f65889/users/38c502ae0355414094fa00b5241d793e/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:24.442530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5caa92c-208c-4bf7-a408-47426b1debe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d918fdf57ad64d1297959c1153f65889, user_id=38c502ae0355414094fa00b5241d793e, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:24.447435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5caa92c-208c-4bf7-a408-47426b1debe2 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:24.479355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1021/2014] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:24 2022] PUT /identity/v3/projects/d918fdf57ad64d1297959c1153f65889/users/38c502ae0355414094fa00b5241d793e/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:24.490952 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e19e2-9851-4c29-8e01-71b3b9075f97 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:24.490952 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e19e2-9851-4c29-8e01-71b3b9075f97 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:24.491234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e19e2-9851-4c29-8e01-71b3b9075f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:24.588964 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 994/2015] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:24.590821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-b25e19e2-9851-4c29-8e01-71b3b9075f97 None None] MFA Rules not processed for user `38c502ae0355414094fa00b5241d793e`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:24.592786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b25e19e2-9851-4c29-8e01-71b3b9075f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:24.603292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76968b54-d07f-4ca5-9c61-50530bf1e123 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:24.619666 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76968b54-d07f-4ca5-9c61-50530bf1e123 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:24.638975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:24.759427 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174baf5-9c1d-4d39-a141-ba36f072f931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:24.760134 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174baf5-9c1d-4d39-a141-ba36f072f931 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:24.760423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174baf5-9c1d-4d39-a141-ba36f072f931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:24.760762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174baf5-9c1d-4d39-a141-ba36f072f931 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:24.761957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174baf5-9c1d-4d39-a141-ba36f072f931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:24.764118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174baf5-9c1d-4d39-a141-ba36f072f931 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:24.787584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1022/2016] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:24.792317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e19e2-9851-4c29-8e01-71b3b9075f97 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:24.802738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 995/2017] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:24 2022] POST /identity/v3/projects => generated 381 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:24.808392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174baf5-9c1d-4d39-a141-ba36f072f931 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:24.819631 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e19e2-9851-4c29-8e01-71b3b9075f97 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:24.877532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:24.902038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174baf5-9c1d-4d39-a141-ba36f072f931 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:24.913611 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33af5d4-e2f5-48ba-9ace-0befb77e90f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:24.914483 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33af5d4-e2f5-48ba-9ace-0befb77e90f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:24.916049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33af5d4-e2f5-48ba-9ace-0befb77e90f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:24.916584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33af5d4-e2f5-48ba-9ace-0befb77e90f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:24.919577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33af5d4-e2f5-48ba-9ace-0befb77e90f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:24.924350 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33af5d4-e2f5-48ba-9ace-0befb77e90f7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:24.974373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3bb09-56ee-44d5-84cf-bb4be287fb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:24.978546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 996/2018] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:24 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:24.978741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3bb09-56ee-44d5-84cf-bb4be287fb06 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:24.978741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3bb09-56ee-44d5-84cf-bb4be287fb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:24.978741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3bb09-56ee-44d5-84cf-bb4be287fb06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:24.978741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3bb09-56ee-44d5-84cf-bb4be287fb06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:24.980109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33af5d4-e2f5-48ba-9ace-0befb77e90f7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:24.988215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3bb09-56ee-44d5-84cf-bb4be287fb06 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:25.009386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33af5d4-e2f5-48ba-9ace-0befb77e90f7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:25.019534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280ca0d-d02e-45be-8395-2c1e4d0b2b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:25.020629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280ca0d-d02e-45be-8395-2c1e4d0b2b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:25.021285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280ca0d-d02e-45be-8395-2c1e4d0b2b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:25.022521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280ca0d-d02e-45be-8395-2c1e4d0b2b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:25.023630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280ca0d-d02e-45be-8395-2c1e4d0b2b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:25.025978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280ca0d-d02e-45be-8395-2c1e4d0b2b4b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:25.046863 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 997/2019] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:25.048775 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280ca0d-d02e-45be-8395-2c1e4d0b2b4b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:25.055773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1023/2020] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:24 2022] POST /identity/v3/projects => generated 379 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:25.058261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3bb09-56ee-44d5-84cf-bb4be287fb06 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:25.101051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280ca0d-d02e-45be-8395-2c1e4d0b2b4b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:25.253302 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3bb09-56ee-44d5-84cf-bb4be287fb06 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:25.275183 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e942c9-a826-4500-8656-b80cb987005e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:25.275929 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e942c9-a826-4500-8656-b80cb987005e admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:25.276226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e942c9-a826-4500-8656-b80cb987005e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:25.276711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e942c9-a826-4500-8656-b80cb987005e admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:25.277570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e942c9-a826-4500-8656-b80cb987005e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:25.279714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e942c9-a826-4500-8656-b80cb987005e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:25.320273 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1024/2021] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:25 2022] POST /identity/v3/users => generated 296 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:25.326334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e942c9-a826-4500-8656-b80cb987005e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:25.339067 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e942c9-a826-4500-8656-b80cb987005e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:25.344719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024fcbf-f147-4ba9-9c83-c543270c99ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:25.345923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024fcbf-f147-4ba9-9c83-c543270c99ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:25.346150 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024fcbf-f147-4ba9-9c83-c543270c99ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:25.346481 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024fcbf-f147-4ba9-9c83-c543270c99ee admin admin] PATH_INFO: `/v3/projects/94acbacaac6a4ba4a3211c8d885c6e69/users/46b1785d87e04cc8a3db77c66f7c032c/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:25.353797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834882b2-3dc4-4500-a07c-b9fc7878ef6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:25.354513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834882b2-3dc4-4500-a07c-b9fc7878ef6e admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:25.354800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834882b2-3dc4-4500-a07c-b9fc7878ef6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:25.355082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834882b2-3dc4-4500-a07c-b9fc7878ef6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:25.359259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834882b2-3dc4-4500-a07c-b9fc7878ef6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:25.359259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834882b2-3dc4-4500-a07c-b9fc7878ef6e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:25.374270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1025/2022] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:25.383710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024fcbf-f147-4ba9-9c83-c543270c99ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=94acbacaac6a4ba4a3211c8d885c6e69, user_id=46b1785d87e04cc8a3db77c66f7c032c, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:25.387235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024fcbf-f147-4ba9-9c83-c543270c99ee admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:25.393799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834882b2-3dc4-4500-a07c-b9fc7878ef6e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:25.415055 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 998/2023] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:25 2022] PUT /identity/v3/projects/94acbacaac6a4ba4a3211c8d885c6e69/users/46b1785d87e04cc8a3db77c66f7c032c/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:25.417623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024fcbf-f147-4ba9-9c83-c543270c99ee admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:25.420426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834882b2-3dc4-4500-a07c-b9fc7878ef6e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:25.452864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:25.523475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb25e1a-ba04-40b8-80b0-00e7c4dbd29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:25.524858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb25e1a-ba04-40b8-80b0-00e7c4dbd29c admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:25.525369 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb25e1a-ba04-40b8-80b0-00e7c4dbd29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:25.525695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb25e1a-ba04-40b8-80b0-00e7c4dbd29c admin admin] PATH_INFO: `/v3/projects/57f12714603e444d805a17ab275c9da9/users/60f41fc129344d7a90e7e3eac9a54ad5/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:25.547402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb25e1a-ba04-40b8-80b0-00e7c4dbd29c admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f12714603e444d805a17ab275c9da9, user_id=60f41fc129344d7a90e7e3eac9a54ad5, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:25.549452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024fcbf-f147-4ba9-9c83-c543270c99ee admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:25.551905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb25e1a-ba04-40b8-80b0-00e7c4dbd29c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:25.556316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc2f8e-16c9-4535-89fd-3938a8024d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:25.556316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc2f8e-16c9-4535-89fd-3938a8024d66 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:25.556316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc2f8e-16c9-4535-89fd-3938a8024d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:25.556737 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc2f8e-16c9-4535-89fd-3938a8024d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:25.557597 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc2f8e-16c9-4535-89fd-3938a8024d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:25.559534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc2f8e-16c9-4535-89fd-3938a8024d66 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:25.574217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 999/2024] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:25.578596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc2f8e-16c9-4535-89fd-3938a8024d66 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:25.579686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1026/2025] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:25 2022] PUT /identity/v3/projects/57f12714603e444d805a17ab275c9da9/users/60f41fc129344d7a90e7e3eac9a54ad5/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:25.583813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb25e1a-ba04-40b8-80b0-00e7c4dbd29c admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:25.593462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc2f8e-16c9-4535-89fd-3938a8024d66 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:25.666381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:25.712221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc853ad-3bae-4163-a43f-37c80e8ac3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:25.713268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc853ad-3bae-4163-a43f-37c80e8ac3e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:25.713631 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc853ad-3bae-4163-a43f-37c80e8ac3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:25.715700 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc853ad-3bae-4163-a43f-37c80e8ac3e4 admin admin] PATH_INFO: `/v3/projects/94acbacaac6a4ba4a3211c8d885c6e69/users/46b1785d87e04cc8a3db77c66f7c032c/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:25.716573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc853ad-3bae-4163-a43f-37c80e8ac3e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94acbacaac6a4ba4a3211c8d885c6e69, user_id=46b1785d87e04cc8a3db77c66f7c032c, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:25.723192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc853ad-3bae-4163-a43f-37c80e8ac3e4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:25.724442 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb25e1a-ba04-40b8-80b0-00e7c4dbd29c admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:25.734245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2c6d5-c073-4cc1-99da-facf55d084a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:25.735040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2c6d5-c073-4cc1-99da-facf55d084a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:25.735210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2c6d5-c073-4cc1-99da-facf55d084a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:25.735498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2c6d5-c073-4cc1-99da-facf55d084a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:25.736436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2c6d5-c073-4cc1-99da-facf55d084a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:25.738123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2c6d5-c073-4cc1-99da-facf55d084a4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:25.760890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1000/2026] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:25 2022] PUT /identity/v3/projects/94acbacaac6a4ba4a3211c8d885c6e69/users/46b1785d87e04cc8a3db77c66f7c032c/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:25.767339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1027/2027] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:37:25.774083 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc853ad-3bae-4163-a43f-37c80e8ac3e4 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:25.781447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42089692-657d-4ca5-ba7a-0b692b5bb960 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:25.781751 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42089692-657d-4ca5-ba7a-0b692b5bb960 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:25.782033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42089692-657d-4ca5-ba7a-0b692b5bb960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:25.824059 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-42089692-657d-4ca5-ba7a-0b692b5bb960 None None] MFA Rules not processed for user `46b1785d87e04cc8a3db77c66f7c032c`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:25.825558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-42089692-657d-4ca5-ba7a-0b692b5bb960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:25.883401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc853ad-3bae-4163-a43f-37c80e8ac3e4 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:25.889543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9042982-7f29-4ff1-8264-4ca995777364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:25.890772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9042982-7f29-4ff1-8264-4ca995777364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:25.891109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9042982-7f29-4ff1-8264-4ca995777364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:25.891390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9042982-7f29-4ff1-8264-4ca995777364 admin admin] PATH_INFO: `/v3/projects/57f12714603e444d805a17ab275c9da9/users/60f41fc129344d7a90e7e3eac9a54ad5/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:25.894460 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9042982-7f29-4ff1-8264-4ca995777364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f12714603e444d805a17ab275c9da9, user_id=60f41fc129344d7a90e7e3eac9a54ad5, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:25.898722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9042982-7f29-4ff1-8264-4ca995777364 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:25.916936 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1001/2028] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:25 2022] PUT /identity/v3/projects/57f12714603e444d805a17ab275c9da9/users/60f41fc129344d7a90e7e3eac9a54ad5/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:25.927189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7192b9-d818-4877-b6d3-0fce88435120 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:25.928336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7192b9-d818-4877-b6d3-0fce88435120 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:25.928336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7192b9-d818-4877-b6d3-0fce88435120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:25.948737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1028/2029] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:25.961797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42089692-657d-4ca5-ba7a-0b692b5bb960 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:25.963250 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-42089692-657d-4ca5-ba7a-0b692b5bb960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:25.966271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ef7192b9-d818-4877-b6d3-0fce88435120 None None] MFA Rules not processed for user `60f41fc129344d7a90e7e3eac9a54ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:25.967472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ef7192b9-d818-4877-b6d3-0fce88435120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:26.008771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:26.065998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42089692-657d-4ca5-ba7a-0b692b5bb960 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:26.093780 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1002/2030] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:26.107995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7192b9-d818-4877-b6d3-0fce88435120 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:26.110977 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ef7192b9-d818-4877-b6d3-0fce88435120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:26.167961 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4dfe1-8f8e-47f8-b605-ccecf046a7c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:26.168534 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4dfe1-8f8e-47f8-b605-ccecf046a7c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:26.168820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4dfe1-8f8e-47f8-b605-ccecf046a7c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:26.169132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4dfe1-8f8e-47f8-b605-ccecf046a7c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:26.246404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7192b9-d818-4877-b6d3-0fce88435120 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:26.250699 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4dfe1-8f8e-47f8-b605-ccecf046a7c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:26.252525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4dfe1-8f8e-47f8-b605-ccecf046a7c9 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:26.254843 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29478b-45e4-4030-a677-11f9be3d5df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:26.255729 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29478b-45e4-4030-a677-11f9be3d5df5 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:26.256545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29478b-45e4-4030-a677-11f9be3d5df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:26.256696 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29478b-45e4-4030-a677-11f9be3d5df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:26.268375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29478b-45e4-4030-a677-11f9be3d5df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:26.269872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29478b-45e4-4030-a677-11f9be3d5df5 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:26.270347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1029/2031] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:37:26.277712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:26.284135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1003/2032] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:26.403763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4dfe1-8f8e-47f8-b605-ccecf046a7c9 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:26.421601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4dfe1-8f8e-47f8-b605-ccecf046a7c9 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:26.424526 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f75dde-da82-4d45-805e-0d55537d1c4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:26.425318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75dde-da82-4d45-805e-0d55537d1c4d service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:26.425530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75dde-da82-4d45-805e-0d55537d1c4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:26.425867 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75dde-da82-4d45-805e-0d55537d1c4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:26.532818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f75dde-da82-4d45-805e-0d55537d1c4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:26.535146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f75dde-da82-4d45-805e-0d55537d1c4d service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:26.546784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1030/2033] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:27.066743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29478b-45e4-4030-a677-11f9be3d5df5 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:27.158018 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29478b-45e4-4030-a677-11f9be3d5df5 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:27.164992 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:27.166100 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:27.166409 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:27.166724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] PATH_INFO: `/v3/users/38c502ae0355414094fa00b5241d793e` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:27.168200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38c502ae0355414094fa00b5241d793e)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:27.170417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:27.242154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38c502ae0355414094fa00b5241d793e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:27.242832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38c502ae0355414094fa00b5241d793e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:27.255116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38c502ae0355414094fa00b5241d793e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:27.256647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38c502ae0355414094fa00b5241d793e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:27.270361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38c502ae0355414094fa00b5241d793e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:27.283867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1004/2034] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:27 2022] DELETE /identity/v3/users/38c502ae0355414094fa00b5241d793e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:27.288612 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b736db-c50d-4d3b-8e77-6d474b7a5339 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:27.289253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b736db-c50d-4d3b-8e77-6d474b7a5339 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:27.289253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b736db-c50d-4d3b-8e77-6d474b7a5339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:27.307774 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-70b736db-c50d-4d3b-8e77-6d474b7a5339 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:27.309010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-70b736db-c50d-4d3b-8e77-6d474b7a5339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:27.337720 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:27.375978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1031/2035] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:27.386397 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:27.387871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:27.454494 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d62fc-f12a-44ad-b703-cc771b822f28 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:27.458416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283cb27-c32d-4818-a4ae-3df3673da4f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:27.459076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283cb27-c32d-4818-a4ae-3df3673da4f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:27.459355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283cb27-c32d-4818-a4ae-3df3673da4f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:27.459628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283cb27-c32d-4818-a4ae-3df3673da4f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:27.470653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283cb27-c32d-4818-a4ae-3df3673da4f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:27.472580 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283cb27-c32d-4818-a4ae-3df3673da4f3 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:27.483028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1005/2036] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:27.830043 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b736db-c50d-4d3b-8e77-6d474b7a5339 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:27.831674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-70b736db-c50d-4d3b-8e77-6d474b7a5339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:27.860746 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b736db-c50d-4d3b-8e77-6d474b7a5339 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:27.866451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:27.867675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:27.867957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:27.868344 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] PATH_INFO: `/v3/projects/d918fdf57ad64d1297959c1153f65889` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:27.869660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d918fdf57ad64d1297959c1153f65889)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:27.871986 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:27.969572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd918fdf57ad64d1297959c1153f65889', 'request_id': 'req-271f79a8-f553-46d8-9b0a-1fc14501c97a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:27.978801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd918fdf57ad64d1297959c1153f65889', 'request_id': 'req-271f79a8-f553-46d8-9b0a-1fc14501c97a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:27.989871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd918fdf57ad64d1297959c1153f65889', 'request_id': 'req-271f79a8-f553-46d8-9b0a-1fc14501c97a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:28.010395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] The token cache is being invalidate because project d918fdf57ad64d1297959c1153f65889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:28.010922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:28.012110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:28.014888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1032/2037] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:27 2022] DELETE /identity/v3/projects/d918fdf57ad64d1297959c1153f65889 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:28.025785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283cb27-c32d-4818-a4ae-3df3673da4f3 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:28.035299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283cb27-c32d-4818-a4ae-3df3673da4f3 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:28.071274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:28.181525 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:28.183319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:28.183804 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:28.184362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] PATH_INFO: `/v3/users/60f41fc129344d7a90e7e3eac9a54ad5` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:28.187279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60f41fc129344d7a90e7e3eac9a54ad5)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:28.192009 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:28.341359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60f41fc129344d7a90e7e3eac9a54ad5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:28.342133 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60f41fc129344d7a90e7e3eac9a54ad5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:28.354366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60f41fc129344d7a90e7e3eac9a54ad5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:28.355073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60f41fc129344d7a90e7e3eac9a54ad5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:28.388942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60f41fc129344d7a90e7e3eac9a54ad5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:28.409501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1006/2038] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:28 2022] DELETE /identity/v3/users/60f41fc129344d7a90e7e3eac9a54ad5 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:29.404421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:29.405959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:29.422257 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f79a8-f553-46d8-9b0a-1fc14501c97a admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:29.424887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:29.426529 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:29.516842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ee269-2239-49df-86df-8519cc43a037 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:29.522829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e13833-7685-43cc-bebe-35fba426353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:29.523745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e13833-7685-43cc-bebe-35fba426353a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:29.524032 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e13833-7685-43cc-bebe-35fba426353a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:29.524315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e13833-7685-43cc-bebe-35fba426353a admin admin] PATH_INFO: `/v3/projects/57f12714603e444d805a17ab275c9da9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:29.525667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e13833-7685-43cc-bebe-35fba426353a admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f12714603e444d805a17ab275c9da9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:29.527738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e13833-7685-43cc-bebe-35fba426353a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:29.544173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11366d85-b963-465d-bfe9-762bb0e002c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:29.547082 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11366d85-b963-465d-bfe9-762bb0e002c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:29.547444 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11366d85-b963-465d-bfe9-762bb0e002c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:29.547754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11366d85-b963-465d-bfe9-762bb0e002c2 admin admin] PATH_INFO: `/v3/users/e61d179002a447d89160d8cb435089eb` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:29.550037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11366d85-b963-465d-bfe9-762bb0e002c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e61d179002a447d89160d8cb435089eb)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:29.554532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11366d85-b963-465d-bfe9-762bb0e002c2 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:29.619099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:29.625722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-90e13833-7685-43cc-bebe-35fba426353a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f12714603e444d805a17ab275c9da9', 'request_id': 'req-90e13833-7685-43cc-bebe-35fba426353a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:29.635290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-90e13833-7685-43cc-bebe-35fba426353a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f12714603e444d805a17ab275c9da9', 'request_id': 'req-90e13833-7685-43cc-bebe-35fba426353a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:29.650801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-90e13833-7685-43cc-bebe-35fba426353a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f12714603e444d805a17ab275c9da9', 'request_id': 'req-90e13833-7685-43cc-bebe-35fba426353a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:29.668046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-90e13833-7685-43cc-bebe-35fba426353a admin admin] The token cache is being invalidate because project 57f12714603e444d805a17ab275c9da9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:29.668360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-11366d85-b963-465d-bfe9-762bb0e002c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e61d179002a447d89160d8cb435089eb'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:29.668745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-90e13833-7685-43cc-bebe-35fba426353a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:29.669220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-11366d85-b963-465d-bfe9-762bb0e002c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e61d179002a447d89160d8cb435089eb'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:29.671000 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-90e13833-7685-43cc-bebe-35fba426353a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:29.673249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1007/2039] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:29 2022] DELETE /identity/v3/projects/57f12714603e444d805a17ab275c9da9 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:29.678574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfd113-dd5e-4566-8f0f-779cc9faa9bc None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:29.678859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfd113-dd5e-4566-8f0f-779cc9faa9bc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:29.679131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfd113-dd5e-4566-8f0f-779cc9faa9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:29.691956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:29.695137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-11366d85-b963-465d-bfe9-762bb0e002c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e61d179002a447d89160d8cb435089eb'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:29.703389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-64cfd113-dd5e-4566-8f0f-779cc9faa9bc None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:29.705398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-64cfd113-dd5e-4566-8f0f-779cc9faa9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:29.715474 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-11366d85-b963-465d-bfe9-762bb0e002c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e61d179002a447d89160d8cb435089eb'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:29.716586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-11366d85-b963-465d-bfe9-762bb0e002c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e61d179002a447d89160d8cb435089eb'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:29.747819 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1033/2040] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:29 2022] DELETE /identity/v3/users/e61d179002a447d89160d8cb435089eb => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:29.825706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1008/2041] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:29.834827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc426ef9-f655-4058-8b85-ac0ae2c025e0 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:29.835288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc426ef9-f655-4058-8b85-ac0ae2c025e0 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:29.835645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc426ef9-f655-4058-8b85-ac0ae2c025e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:29.869399 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-bc426ef9-f655-4058-8b85-ac0ae2c025e0 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:29.870985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-bc426ef9-f655-4058-8b85-ac0ae2c025e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:29.994749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1034/2042] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:30.000759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfd113-dd5e-4566-8f0f-779cc9faa9bc None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.012954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfd113-dd5e-4566-8f0f-779cc9faa9bc None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.018545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62861bb-2cc4-4a78-bd48-3b9d948ad33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.019268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62861bb-2cc4-4a78-bd48-3b9d948ad33d admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.019557 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62861bb-2cc4-4a78-bd48-3b9d948ad33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.019836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62861bb-2cc4-4a78-bd48-3b9d948ad33d admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.020608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62861bb-2cc4-4a78-bd48-3b9d948ad33d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:37:30.020870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62861bb-2cc4-4a78-bd48-3b9d948ad33d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.022629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62861bb-2cc4-4a78-bd48-3b9d948ad33d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.042126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1009/2043] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:37:30.047115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc426ef9-f655-4058-8b85-ac0ae2c025e0 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.068731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc426ef9-f655-4058-8b85-ac0ae2c025e0 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.074501 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ab083-b44e-4424-b344-b3d1dfa4d096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.075855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ab083-b44e-4424-b344-b3d1dfa4d096 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.076558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ab083-b44e-4424-b344-b3d1dfa4d096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.076558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ab083-b44e-4424-b344-b3d1dfa4d096 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.077400 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ab083-b44e-4424-b344-b3d1dfa4d096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.079570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ab083-b44e-4424-b344-b3d1dfa4d096 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.109364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62861bb-2cc4-4a78-bd48-3b9d948ad33d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.111808 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f62861bb-2cc4-4a78-bd48-3b9d948ad33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:30.133108 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62861bb-2cc4-4a78-bd48-3b9d948ad33d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.136678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1035/2044] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:30 2022] POST /identity/v3/projects => generated 361 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:30.141690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ab083-b44e-4424-b344-b3d1dfa4d096 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.208755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4abf4e8-73ba-4b3a-b09b-e88999f24174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.210570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf4e8-73ba-4b3a-b09b-e88999f24174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.210973 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf4e8-73ba-4b3a-b09b-e88999f24174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.211152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf4e8-73ba-4b3a-b09b-e88999f24174 admin admin] PATH_INFO: `/v3/projects/973dc13b7788475bbe4cff5dac7b873e` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.212776 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4abf4e8-73ba-4b3a-b09b-e88999f24174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=973dc13b7788475bbe4cff5dac7b873e)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.214750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4abf4e8-73ba-4b3a-b09b-e88999f24174 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.225785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ab083-b44e-4424-b344-b3d1dfa4d096 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.232417 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e25101-de36-48b2-8504-40a91053fa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.233426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e25101-de36-48b2-8504-40a91053fa56 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.233736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e25101-de36-48b2-8504-40a91053fa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.234225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e25101-de36-48b2-8504-40a91053fa56 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.234962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e25101-de36-48b2-8504-40a91053fa56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.237230 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e25101-de36-48b2-8504-40a91053fa56 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.264219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1036/2045] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:30 2022] POST /identity/v3/users => generated 281 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:30.269754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e25101-de36-48b2-8504-40a91053fa56 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.280769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e25101-de36-48b2-8504-40a91053fa56 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.282658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e4abf4e8-73ba-4b3a-b09b-e88999f24174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '973dc13b7788475bbe4cff5dac7b873e', 'request_id': 'req-e4abf4e8-73ba-4b3a-b09b-e88999f24174'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:30.290618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e4abf4e8-73ba-4b3a-b09b-e88999f24174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '973dc13b7788475bbe4cff5dac7b873e', 'request_id': 'req-e4abf4e8-73ba-4b3a-b09b-e88999f24174'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:30.295349 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:30.307575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e4abf4e8-73ba-4b3a-b09b-e88999f24174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '973dc13b7788475bbe4cff5dac7b873e', 'request_id': 'req-e4abf4e8-73ba-4b3a-b09b-e88999f24174'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:30.321373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e4abf4e8-73ba-4b3a-b09b-e88999f24174 admin admin] The token cache is being invalidate because project 973dc13b7788475bbe4cff5dac7b873e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:30.321748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e4abf4e8-73ba-4b3a-b09b-e88999f24174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:30.322429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e4abf4e8-73ba-4b3a-b09b-e88999f24174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:30.324089 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1010/2046] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:30 2022] DELETE /identity/v3/projects/973dc13b7788475bbe4cff5dac7b873e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:30.331083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:30.331303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209352e-abd0-4578-8642-d680f1b6c979 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.331495 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209352e-abd0-4578-8642-d680f1b6c979 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.331776 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209352e-abd0-4578-8642-d680f1b6c979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.353530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e209352e-abd0-4578-8642-d680f1b6c979 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:30.354711 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e209352e-abd0-4578-8642-d680f1b6c979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:30.357281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbe746-3024-48aa-8d74-babd0f348eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.357976 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbe746-3024-48aa-8d74-babd0f348eec admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.358377 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbe746-3024-48aa-8d74-babd0f348eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.358807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbe746-3024-48aa-8d74-babd0f348eec admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.360238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dbe746-3024-48aa-8d74-babd0f348eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.362494 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dbe746-3024-48aa-8d74-babd0f348eec admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.367105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1011/2047] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:30.374820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096e49c-ef1b-4340-aa2a-c465c436e871 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.375224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096e49c-ef1b-4340-aa2a-c465c436e871 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.375602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096e49c-ef1b-4340-aa2a-c465c436e871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.375744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1037/2048] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:30.379601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbe746-3024-48aa-8d74-babd0f348eec admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.381883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c3dbe746-3024-48aa-8d74-babd0f348eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:30.394630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbe746-3024-48aa-8d74-babd0f348eec admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.396222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-6096e49c-ef1b-4340-aa2a-c465c436e871 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:30.397876 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6096e49c-ef1b-4340-aa2a-c465c436e871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:30.400343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac886d6-7688-4b00-a315-2e38c86980c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.401019 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac886d6-7688-4b00-a315-2e38c86980c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.401328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac886d6-7688-4b00-a315-2e38c86980c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.401610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac886d6-7688-4b00-a315-2e38c86980c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.403077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac886d6-7688-4b00-a315-2e38c86980c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.404796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac886d6-7688-4b00-a315-2e38c86980c7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.407971 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1012/2049] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:30.412166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096e49c-ef1b-4340-aa2a-c465c436e871 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.420264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1038/2050] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:37:30.421662 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096e49c-ef1b-4340-aa2a-c465c436e871 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.424455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac886d6-7688-4b00-a315-2e38c86980c7 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.427273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422f527-22cd-4a75-89e1-72cfde5cc607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.428741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422f527-22cd-4a75-89e1-72cfde5cc607 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.429032 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422f527-22cd-4a75-89e1-72cfde5cc607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.429499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422f527-22cd-4a75-89e1-72cfde5cc607 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.430239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422f527-22cd-4a75-89e1-72cfde5cc607 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:37:30.430546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422f527-22cd-4a75-89e1-72cfde5cc607 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.432170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422f527-22cd-4a75-89e1-72cfde5cc607 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.438151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac886d6-7688-4b00-a315-2e38c86980c7 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.444586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dcc5c0-5402-4dfa-b4af-48b39e80fae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.445528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dcc5c0-5402-4dfa-b4af-48b39e80fae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.445846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dcc5c0-5402-4dfa-b4af-48b39e80fae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.446328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dcc5c0-5402-4dfa-b4af-48b39e80fae0 admin admin] PATH_INFO: `/v3/projects/0dc7776ff4d445f9b6647490c9556c50/users/b89c8f0b28d84e46b46be96d487aa5f4/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.446492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1013/2051] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:37:30.450998 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422f527-22cd-4a75-89e1-72cfde5cc607 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.462489 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422f527-22cd-4a75-89e1-72cfde5cc607 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.466801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dcc5c0-5402-4dfa-b4af-48b39e80fae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc7776ff4d445f9b6647490c9556c50, user_id=b89c8f0b28d84e46b46be96d487aa5f4, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.468330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ca821-3662-4c9d-b550-8115f83e3b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.469011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ca821-3662-4c9d-b550-8115f83e3b94 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.469321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ca821-3662-4c9d-b550-8115f83e3b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.469604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ca821-3662-4c9d-b550-8115f83e3b94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.470308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ca821-3662-4c9d-b550-8115f83e3b94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.471388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dcc5c0-5402-4dfa-b4af-48b39e80fae0 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.472557 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ca821-3662-4c9d-b550-8115f83e3b94 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.490879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1039/2052] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:30 2022] PUT /identity/v3/projects/0dc7776ff4d445f9b6647490c9556c50/users/b89c8f0b28d84e46b46be96d487aa5f4/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:30.510402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6c22c-e19d-4ba6-ba5a-965435f8a968 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.510758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6c22c-e19d-4ba6-ba5a-965435f8a968 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.511865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6c22c-e19d-4ba6-ba5a-965435f8a968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.525154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1014/2053] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:30 2022] POST /identity/v3/projects => generated 387 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:37:30.528450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ca821-3662-4c9d-b550-8115f83e3b94 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.547879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-7bf6c22c-e19d-4ba6-ba5a-965435f8a968 None None] MFA Rules not processed for user `b89c8f0b28d84e46b46be96d487aa5f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:30.549492 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-7bf6c22c-e19d-4ba6-ba5a-965435f8a968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:30.603665 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ca821-3662-4c9d-b550-8115f83e3b94 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.608702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511d9b2-b49d-4a1e-829c-b3de5dae0f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.609424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511d9b2-b49d-4a1e-829c-b3de5dae0f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.609712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511d9b2-b49d-4a1e-829c-b3de5dae0f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.609990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511d9b2-b49d-4a1e-829c-b3de5dae0f7f admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.610697 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511d9b2-b49d-4a1e-829c-b3de5dae0f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.612754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511d9b2-b49d-4a1e-829c-b3de5dae0f7f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.627278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1040/2054] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:30.644853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1015/2055] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:30 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:37:30.649280 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6c22c-e19d-4ba6-ba5a-965435f8a968 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.663925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511d9b2-b49d-4a1e-829c-b3de5dae0f7f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.665671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6c22c-e19d-4ba6-ba5a-965435f8a968 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.665894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6511d9b2-b49d-4a1e-829c-b3de5dae0f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:30.671292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f47ad-8332-491c-8e73-8166883d72f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.672296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f47ad-8332-491c-8e73-8166883d72f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.672753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f47ad-8332-491c-8e73-8166883d72f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.673228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f47ad-8332-491c-8e73-8166883d72f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.674380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f47ad-8332-491c-8e73-8166883d72f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.676534 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f47ad-8332-491c-8e73-8166883d72f7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.689615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1041/2056] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:30.698669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f47ad-8332-491c-8e73-8166883d72f7 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.718571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f47ad-8332-491c-8e73-8166883d72f7 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.726731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc55a7-1268-4268-9c70-65d8bb86e0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.727819 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc55a7-1268-4268-9c70-65d8bb86e0ee admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.727819 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc55a7-1268-4268-9c70-65d8bb86e0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.728307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc55a7-1268-4268-9c70-65d8bb86e0ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.729524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc55a7-1268-4268-9c70-65d8bb86e0ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.731003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc55a7-1268-4268-9c70-65d8bb86e0ee admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.746865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1042/2057] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:30.748458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc55a7-1268-4268-9c70-65d8bb86e0ee admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.760989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc55a7-1268-4268-9c70-65d8bb86e0ee admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.766873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511d9b2-b49d-4a1e-829c-b3de5dae0f7f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.771324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b2547-9598-4eb1-9f73-13ffcea7ceb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.772186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959393bc-acd0-40d7-aa2d-273986cabe20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.773174 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959393bc-acd0-40d7-aa2d-273986cabe20 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.773306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b2547-9598-4eb1-9f73-13ffcea7ceb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.773555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b2547-9598-4eb1-9f73-13ffcea7ceb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.773649 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959393bc-acd0-40d7-aa2d-273986cabe20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.773838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b2547-9598-4eb1-9f73-13ffcea7ceb9 admin admin] PATH_INFO: `/v3/projects/cc6d788a2d5d4a5c9f559836207e8d62/users/876e3626cae145048dd1dbb4650a5603/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.774046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959393bc-acd0-40d7-aa2d-273986cabe20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.785674 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959393bc-acd0-40d7-aa2d-273986cabe20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.787887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959393bc-acd0-40d7-aa2d-273986cabe20 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.793144 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b2547-9598-4eb1-9f73-13ffcea7ceb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc6d788a2d5d4a5c9f559836207e8d62, user_id=876e3626cae145048dd1dbb4650a5603, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.795946 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b2547-9598-4eb1-9f73-13ffcea7ceb9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.801644 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1016/2058] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:37:30.820699 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1043/2059] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:30 2022] PUT /identity/v3/projects/cc6d788a2d5d4a5c9f559836207e8d62/users/876e3626cae145048dd1dbb4650a5603/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:30.822486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959393bc-acd0-40d7-aa2d-273986cabe20 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.827885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2ca8d-6a3f-40f5-843d-25cecffe6d13 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.827885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2ca8d-6a3f-40f5-843d-25cecffe6d13 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.827885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2ca8d-6a3f-40f5-843d-25cecffe6d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.838866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959393bc-acd0-40d7-aa2d-273986cabe20 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:30.849593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-efc2ca8d-6a3f-40f5-843d-25cecffe6d13 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:30.851726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-efc2ca8d-6a3f-40f5-843d-25cecffe6d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:30.886508 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:30.917126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:30.927898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedd663-cbd4-4d74-97ca-ea1bf9a41007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:30.929402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedd663-cbd4-4d74-97ca-ea1bf9a41007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:30.933504 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedd663-cbd4-4d74-97ca-ea1bf9a41007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:30.933794 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedd663-cbd4-4d74-97ca-ea1bf9a41007 admin admin] PATH_INFO: `/v3/users/b89c8f0b28d84e46b46be96d487aa5f4` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:30.935178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedd663-cbd4-4d74-97ca-ea1bf9a41007 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b89c8f0b28d84e46b46be96d487aa5f4)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:30.937276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedd663-cbd4-4d74-97ca-ea1bf9a41007 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:30.967344 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1044/2060] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:30.978451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2ca8d-6a3f-40f5-843d-25cecffe6d13 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:30.980554 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-efc2ca8d-6a3f-40f5-843d-25cecffe6d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:31.029640 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dfedd663-cbd4-4d74-97ca-ea1bf9a41007 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b89c8f0b28d84e46b46be96d487aa5f4'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:31.030413 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dfedd663-cbd4-4d74-97ca-ea1bf9a41007 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b89c8f0b28d84e46b46be96d487aa5f4'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:31.041709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dfedd663-cbd4-4d74-97ca-ea1bf9a41007 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b89c8f0b28d84e46b46be96d487aa5f4'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:31.042752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dfedd663-cbd4-4d74-97ca-ea1bf9a41007 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b89c8f0b28d84e46b46be96d487aa5f4'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:31.057766 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dfedd663-cbd4-4d74-97ca-ea1bf9a41007 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b89c8f0b28d84e46b46be96d487aa5f4'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:31.060276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2ca8d-6a3f-40f5-843d-25cecffe6d13 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:31.069369 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaa01d-4a93-4ede-a909-9a7582e86dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:31.070272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaa01d-4a93-4ede-a909-9a7582e86dff service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:31.070555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaa01d-4a93-4ede-a909-9a7582e86dff service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:31.070830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaa01d-4a93-4ede-a909-9a7582e86dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:31.073145 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1017/2061] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:30 2022] DELETE /identity/v3/users/b89c8f0b28d84e46b46be96d487aa5f4 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:31.073303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5cdaa01d-4a93-4ede-a909-9a7582e86dff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:31.077609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c5ebc-a17b-4505-916a-33c15c87f993 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:31.078039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c5ebc-a17b-4505-916a-33c15c87f993 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:31.078393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c5ebc-a17b-4505-916a-33c15c87f993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:31.095830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdaa01d-4a93-4ede-a909-9a7582e86dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:31.097402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdaa01d-4a93-4ede-a909-9a7582e86dff service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:31.116817 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1045/2062] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:31.118462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ab1c5ebc-a17b-4505-916a-33c15c87f993 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:31.119273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ab1c5ebc-a17b-4505-916a-33c15c87f993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:31.221459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1018/2063] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:31.228566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaa01d-4a93-4ede-a909-9a7582e86dff service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:31.230448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5cdaa01d-4a93-4ede-a909-9a7582e86dff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:31.250953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaa01d-4a93-4ede-a909-9a7582e86dff service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:31.320180 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fccce-6deb-4ed0-a3e6-c9d7e34fa7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:31.320869 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fccce-6deb-4ed0-a3e6-c9d7e34fa7df service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:31.321180 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fccce-6deb-4ed0-a3e6-c9d7e34fa7df service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:31.321463 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fccce-6deb-4ed0-a3e6-c9d7e34fa7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:31.331575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01fccce-6deb-4ed0-a3e6-c9d7e34fa7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:31.333529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01fccce-6deb-4ed0-a3e6-c9d7e34fa7df service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:31.345963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1046/2064] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:31.971779 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c5ebc-a17b-4505-916a-33c15c87f993 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:31.974160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ab1c5ebc-a17b-4505-916a-33c15c87f993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:32.004777 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c5ebc-a17b-4505-916a-33c15c87f993 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:32.017370 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b6452-318b-4838-bb3f-5844f886f163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:32.019157 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b6452-318b-4838-bb3f-5844f886f163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:32.019663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b6452-318b-4838-bb3f-5844f886f163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:32.020174 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b6452-318b-4838-bb3f-5844f886f163 admin admin] PATH_INFO: `/v3/projects/0dc7776ff4d445f9b6647490c9556c50` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:32.022445 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b6452-318b-4838-bb3f-5844f886f163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc7776ff4d445f9b6647490c9556c50)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:32.026437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b6452-318b-4838-bb3f-5844f886f163 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:32.114480 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:32.122822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-533b6452-318b-4838-bb3f-5844f886f163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc7776ff4d445f9b6647490c9556c50', 'request_id': 'req-533b6452-318b-4838-bb3f-5844f886f163'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:32.130616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-533b6452-318b-4838-bb3f-5844f886f163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc7776ff4d445f9b6647490c9556c50', 'request_id': 'req-533b6452-318b-4838-bb3f-5844f886f163'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:32.141588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-533b6452-318b-4838-bb3f-5844f886f163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc7776ff4d445f9b6647490c9556c50', 'request_id': 'req-533b6452-318b-4838-bb3f-5844f886f163'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:32.185510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-533b6452-318b-4838-bb3f-5844f886f163 admin admin] The token cache is being invalidate because project 0dc7776ff4d445f9b6647490c9556c50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:32.186306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-533b6452-318b-4838-bb3f-5844f886f163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:32.187736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-533b6452-318b-4838-bb3f-5844f886f163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:32.193360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1019/2065] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:31 2022] DELETE /identity/v3/projects/0dc7776ff4d445f9b6647490c9556c50 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:32.199829 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6a48a-c52d-4764-b26a-0877ed010529 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:32.200764 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6a48a-c52d-4764-b26a-0877ed010529 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:32.201220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6a48a-c52d-4764-b26a-0877ed010529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:32.273319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-9cc6a48a-c52d-4764-b26a-0877ed010529 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:32.275029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9cc6a48a-c52d-4764-b26a-0877ed010529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:32.363131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1047/2066] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:32.373435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc6fdd-a829-4042-9d57-fc69c59d92a6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:32.373703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc6fdd-a829-4042-9d57-fc69c59d92a6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:32.374702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc6fdd-a829-4042-9d57-fc69c59d92a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:32.406038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-37bc6fdd-a829-4042-9d57-fc69c59d92a6 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:32.407289 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-37bc6fdd-a829-4042-9d57-fc69c59d92a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:32.422105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1020/2067] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:32.427208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6a48a-c52d-4764-b26a-0877ed010529 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:32.443680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6a48a-c52d-4764-b26a-0877ed010529 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:32.450386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6db360-797d-4d23-a65d-181d5b6df078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:32.451453 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6db360-797d-4d23-a65d-181d5b6df078 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:32.451758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6db360-797d-4d23-a65d-181d5b6df078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:32.452096 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6db360-797d-4d23-a65d-181d5b6df078 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:32.452882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6db360-797d-4d23-a65d-181d5b6df078 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:37:32.453410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6db360-797d-4d23-a65d-181d5b6df078 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:32.455149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6db360-797d-4d23-a65d-181d5b6df078 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:32.478508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1048/2068] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:37:32.479663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc6fdd-a829-4042-9d57-fc69c59d92a6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:32.491374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc6fdd-a829-4042-9d57-fc69c59d92a6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:32.496787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a817403-a888-4765-a2f3-816178b7f263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:32.497645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a817403-a888-4765-a2f3-816178b7f263 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:32.497987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a817403-a888-4765-a2f3-816178b7f263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:32.498378 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a817403-a888-4765-a2f3-816178b7f263 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:32.499131 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a817403-a888-4765-a2f3-816178b7f263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:32.501651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a817403-a888-4765-a2f3-816178b7f263 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:32.534148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1021/2069] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:32 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:32.539734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6db360-797d-4d23-a65d-181d5b6df078 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:32.558740 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6db360-797d-4d23-a65d-181d5b6df078 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:32.580939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:32.648255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e40c64-b9ea-46b7-9f60-5df1de71ab52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:32.649172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e40c64-b9ea-46b7-9f60-5df1de71ab52 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:32.649615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e40c64-b9ea-46b7-9f60-5df1de71ab52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:32.649948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e40c64-b9ea-46b7-9f60-5df1de71ab52 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:32.650782 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e40c64-b9ea-46b7-9f60-5df1de71ab52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:32.652851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e40c64-b9ea-46b7-9f60-5df1de71ab52 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:32.684707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1049/2070] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:32 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:32.690091 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a817403-a888-4765-a2f3-816178b7f263 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:32.703363 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a817403-a888-4765-a2f3-816178b7f263 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:32.709630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88b77f-0a50-474e-ae5b-478823177e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:32.710329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88b77f-0a50-474e-ae5b-478823177e99 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:32.710620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88b77f-0a50-474e-ae5b-478823177e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:32.710901 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88b77f-0a50-474e-ae5b-478823177e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:32.711694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88b77f-0a50-474e-ae5b-478823177e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:32.713347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88b77f-0a50-474e-ae5b-478823177e99 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:32.727839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1022/2071] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:32.735504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e40c64-b9ea-46b7-9f60-5df1de71ab52 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:32.754282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e40c64-b9ea-46b7-9f60-5df1de71ab52 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:32.760332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274b160-3daa-4045-b60e-00b983614183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:32.761049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274b160-3daa-4045-b60e-00b983614183 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:32.761443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274b160-3daa-4045-b60e-00b983614183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:32.761722 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274b160-3daa-4045-b60e-00b983614183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:32.762734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274b160-3daa-4045-b60e-00b983614183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:32.764107 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274b160-3daa-4045-b60e-00b983614183 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:32.777926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1050/2072] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:32.781975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88b77f-0a50-474e-ae5b-478823177e99 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:32.793704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88b77f-0a50-474e-ae5b-478823177e99 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:32.799305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c0ba1-a158-458f-87ba-ea934ec7195d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:32.800198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c0ba1-a158-458f-87ba-ea934ec7195d admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:32.800473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c0ba1-a158-458f-87ba-ea934ec7195d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:32.800747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c0ba1-a158-458f-87ba-ea934ec7195d admin admin] PATH_INFO: `/v3/projects/beb0569df2334722bfd613bb73354ba2/users/3217517e0d5c4eeeab2e96f120a8686b/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:32.820160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c0ba1-a158-458f-87ba-ea934ec7195d admin admin] RBAC: Authorizing `identity:create_grant(project_id=beb0569df2334722bfd613bb73354ba2, user_id=3217517e0d5c4eeeab2e96f120a8686b, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:32.822989 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c0ba1-a158-458f-87ba-ea934ec7195d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:32.844672 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1023/2073] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:32 2022] PUT /identity/v3/projects/beb0569df2334722bfd613bb73354ba2/users/3217517e0d5c4eeeab2e96f120a8686b/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:32.856536 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4b515-6ecd-4cf8-bd21-7e958c623907 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:32.856824 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4b515-6ecd-4cf8-bd21-7e958c623907 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:32.857290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4b515-6ecd-4cf8-bd21-7e958c623907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:32.898248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-afe4b515-6ecd-4cf8-bd21-7e958c623907 None None] MFA Rules not processed for user `3217517e0d5c4eeeab2e96f120a8686b`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:32.899787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-afe4b515-6ecd-4cf8-bd21-7e958c623907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:33.009175 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1051/2074] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:33.020844 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c0ba1-a158-458f-87ba-ea934ec7195d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:33.024033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a21c0ba1-a158-458f-87ba-ea934ec7195d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:33.128524 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c0ba1-a158-458f-87ba-ea934ec7195d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:33.135056 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908f6ed-d8c8-4643-bfe0-5a528f47c648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:33.135907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908f6ed-d8c8-4643-bfe0-5a528f47c648 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:33.136314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908f6ed-d8c8-4643-bfe0-5a528f47c648 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:33.136695 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908f6ed-d8c8-4643-bfe0-5a528f47c648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:33.148763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908f6ed-d8c8-4643-bfe0-5a528f47c648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:33.150956 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908f6ed-d8c8-4643-bfe0-5a528f47c648 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:33.179092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1024/2075] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:37.088876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4b515-6ecd-4cf8-bd21-7e958c623907 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:37.207456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4b515-6ecd-4cf8-bd21-7e958c623907 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:37.217056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e197c-c220-400b-a188-8c984f550cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:37.218642 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e197c-c220-400b-a188-8c984f550cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:37.219226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e197c-c220-400b-a188-8c984f550cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:37.219898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e197c-c220-400b-a188-8c984f550cc8 admin admin] PATH_INFO: `/v3/users/3217517e0d5c4eeeab2e96f120a8686b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:37.221889 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e197c-c220-400b-a188-8c984f550cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3217517e0d5c4eeeab2e96f120a8686b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:37.230338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e197c-c220-400b-a188-8c984f550cc8 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:37.277626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc654d-347a-404f-8144-2a937ac2ef41 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:37.278252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc654d-347a-404f-8144-2a937ac2ef41 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:37.278252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc654d-347a-404f-8144-2a937ac2ef41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:37.284073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:37.319091 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5c4e197c-c220-400b-a188-8c984f550cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3217517e0d5c4eeeab2e96f120a8686b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:37.320076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5c4e197c-c220-400b-a188-8c984f550cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3217517e0d5c4eeeab2e96f120a8686b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:37.327538 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d3fc654d-347a-404f-8144-2a937ac2ef41 None None] MFA Rules not processed for user `876e3626cae145048dd1dbb4650a5603`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:37.327862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5c4e197c-c220-400b-a188-8c984f550cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3217517e0d5c4eeeab2e96f120a8686b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:37.328894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d3fc654d-347a-404f-8144-2a937ac2ef41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:37.352506 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5c4e197c-c220-400b-a188-8c984f550cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3217517e0d5c4eeeab2e96f120a8686b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:37.354056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5c4e197c-c220-400b-a188-8c984f550cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3217517e0d5c4eeeab2e96f120a8686b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:37.379193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1052/2076] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:37 2022] DELETE /identity/v3/users/3217517e0d5c4eeeab2e96f120a8686b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:37.384646 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aeb968-10f9-4268-84e6-133aceebbc35 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:37.385035 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aeb968-10f9-4268-84e6-133aceebbc35 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:37.385410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aeb968-10f9-4268-84e6-133aceebbc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:37.413407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-94aeb968-10f9-4268-84e6-133aceebbc35 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:37.414794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-94aeb968-10f9-4268-84e6-133aceebbc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:37.421758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1025/2077] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:37.437028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc654d-347a-404f-8144-2a937ac2ef41 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:37.438813 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d3fc654d-347a-404f-8144-2a937ac2ef41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:37.464368 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:37.507297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1053/2078] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:37.522816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aeb968-10f9-4268-84e6-133aceebbc35 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:37.526299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-94aeb968-10f9-4268-84e6-133aceebbc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:37.587852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc654d-347a-404f-8144-2a937ac2ef41 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:37.631126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aeb968-10f9-4268-84e6-133aceebbc35 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:37.638518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f60f0-4fd1-4873-b703-e256639e7645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:37.639648 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f60f0-4fd1-4873-b703-e256639e7645 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:37.640275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f60f0-4fd1-4873-b703-e256639e7645 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:37.640607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f60f0-4fd1-4873-b703-e256639e7645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:37.656550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f60f0-4fd1-4873-b703-e256639e7645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:37.659315 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f60f0-4fd1-4873-b703-e256639e7645 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:37.673364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1054/2079] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:37.705967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f60f0-4fd1-4873-b703-e256639e7645 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:37.707516 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-915f60f0-4fd1-4873-b703-e256639e7645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:37.710488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b70ba-575e-49fe-9eb7-89e099d2d244 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:37.712036 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b70ba-575e-49fe-9eb7-89e099d2d244 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:37.712542 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b70ba-575e-49fe-9eb7-89e099d2d244 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:37.713528 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b70ba-575e-49fe-9eb7-89e099d2d244 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:37.724219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f60f0-4fd1-4873-b703-e256639e7645 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:37.732004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:37.732290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:37.732542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:37.732817 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] PATH_INFO: `/v3/users/a1b1f7ccbb09402eb057f2c6e5af7ed1` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:37.739503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b1f7ccbb09402eb057f2c6e5af7ed1)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:37.741797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:37.822919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b70ba-575e-49fe-9eb7-89e099d2d244 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:37.825502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b70ba-575e-49fe-9eb7-89e099d2d244 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:37.836759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:37.838742 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1026/2080] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:37.840731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b1f7ccbb09402eb057f2c6e5af7ed1'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:37.841929 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b1f7ccbb09402eb057f2c6e5af7ed1'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:37.860010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b1f7ccbb09402eb057f2c6e5af7ed1'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:37.884270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b1f7ccbb09402eb057f2c6e5af7ed1'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:37.884993 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b1f7ccbb09402eb057f2c6e5af7ed1'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:37.928144 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1055/2081] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:37 2022] DELETE /identity/v3/users/a1b1f7ccbb09402eb057f2c6e5af7ed1 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:37.936631 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd2540-bd2a-4015-9450-1bd7ed808146 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:37.936967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd2540-bd2a-4015-9450-1bd7ed808146 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:37.937812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd2540-bd2a-4015-9450-1bd7ed808146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:37.992827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a3dd2540-bd2a-4015-9450-1bd7ed808146 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:37.995204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a3dd2540-bd2a-4015-9450-1bd7ed808146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:38.074472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:38.120934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1027/2082] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:38.142646 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:38.144939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:38.270673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a934253-5f64-43a6-b48c-0f11ce39333b admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:38.288828 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf57410-6262-42e2-a056-b344eda4e156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:38.289727 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf57410-6262-42e2-a056-b344eda4e156 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:38.290019 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf57410-6262-42e2-a056-b344eda4e156 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:38.290327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf57410-6262-42e2-a056-b344eda4e156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:38.304189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf57410-6262-42e2-a056-b344eda4e156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:38.306368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf57410-6262-42e2-a056-b344eda4e156 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:38.327606 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1056/2083] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:37:38.448035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd2540-bd2a-4015-9450-1bd7ed808146 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:38.449683 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a3dd2540-bd2a-4015-9450-1bd7ed808146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:38.464533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd2540-bd2a-4015-9450-1bd7ed808146 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:38.469882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:38.470917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:38.471206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:38.471482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] PATH_INFO: `/v3/projects/beb0569df2334722bfd613bb73354ba2` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:38.472727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] RBAC: Authorizing `identity:delete_project(project_id=beb0569df2334722bfd613bb73354ba2)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:38.475707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:38.564089 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beb0569df2334722bfd613bb73354ba2', 'request_id': 'req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:38.570927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beb0569df2334722bfd613bb73354ba2', 'request_id': 'req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:38.582301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beb0569df2334722bfd613bb73354ba2', 'request_id': 'req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:38.604168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] The token cache is being invalidate because project beb0569df2334722bfd613bb73354ba2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:38.604615 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:38.605487 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:38.607446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1028/2084] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:38 2022] DELETE /identity/v3/projects/beb0569df2334722bfd613bb73354ba2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:39.006438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf57410-6262-42e2-a056-b344eda4e156 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:39.008560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-fbf57410-6262-42e2-a056-b344eda4e156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:39.037547 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf57410-6262-42e2-a056-b344eda4e156 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:39.144551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:39.146493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:39.146693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:39.147190 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] PATH_INFO: `/v3/projects/a6394338ffb2436ab6237cfb2f186e8b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:39.149067 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6394338ffb2436ab6237cfb2f186e8b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:39.153304 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:39.220108 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6394338ffb2436ab6237cfb2f186e8b', 'request_id': 'req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:39.229154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6394338ffb2436ab6237cfb2f186e8b', 'request_id': 'req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:39.245593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6394338ffb2436ab6237cfb2f186e8b', 'request_id': 'req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:39.263711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:39.283788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] The token cache is being invalidate because project a6394338ffb2436ab6237cfb2f186e8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:39.284223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:39.285375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:39.295905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1057/2085] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:39 2022] DELETE /identity/v3/projects/a6394338ffb2436ab6237cfb2f186e8b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:39.304580 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:39.306914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:39.366077 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266db4fb-cef2-4f00-9bd9-f0029c8ad70c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:39.620719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730a877-4943-440c-abb1-973be195f2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:39.622514 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730a877-4943-440c-abb1-973be195f2ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:39.622970 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730a877-4943-440c-abb1-973be195f2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:39.623515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730a877-4943-440c-abb1-973be195f2ef admin admin] PATH_INFO: `/v3/users/46b1785d87e04cc8a3db77c66f7c032c` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:39.626781 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730a877-4943-440c-abb1-973be195f2ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=46b1785d87e04cc8a3db77c66f7c032c)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:39.629658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730a877-4943-440c-abb1-973be195f2ef admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:39.707743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:39.734759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2730a877-4943-440c-abb1-973be195f2ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46b1785d87e04cc8a3db77c66f7c032c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:39.735628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2730a877-4943-440c-abb1-973be195f2ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46b1785d87e04cc8a3db77c66f7c032c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:39.749840 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2730a877-4943-440c-abb1-973be195f2ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46b1785d87e04cc8a3db77c66f7c032c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:39.750666 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2730a877-4943-440c-abb1-973be195f2ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46b1785d87e04cc8a3db77c66f7c032c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:39.769245 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2730a877-4943-440c-abb1-973be195f2ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46b1785d87e04cc8a3db77c66f7c032c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:39.796344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1029/2086] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:39 2022] DELETE /identity/v3/users/46b1785d87e04cc8a3db77c66f7c032c => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:40.293123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:40.294878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:40.396634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc2dd1-a0ea-42b9-99ea-8d5cc5b9b9d7 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:40.528291 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c993769-3cbc-46f4-bf07-4a98ba8e1301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:40.530871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c993769-3cbc-46f4-bf07-4a98ba8e1301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:40.531265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c993769-3cbc-46f4-bf07-4a98ba8e1301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:40.531699 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c993769-3cbc-46f4-bf07-4a98ba8e1301 admin admin] PATH_INFO: `/v3/projects/94acbacaac6a4ba4a3211c8d885c6e69` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:40.534355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c993769-3cbc-46f4-bf07-4a98ba8e1301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94acbacaac6a4ba4a3211c8d885c6e69)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:40.537936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c993769-3cbc-46f4-bf07-4a98ba8e1301 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:40.600266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce558c1a-8559-4e31-908f-a2e71f400722 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:40.602201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce558c1a-8559-4e31-908f-a2e71f400722 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:40.602821 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce558c1a-8559-4e31-908f-a2e71f400722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:40.627240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ce558c1a-8559-4e31-908f-a2e71f400722 None None] MFA Rules not processed for user `876e3626cae145048dd1dbb4650a5603`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:40.631516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ce558c1a-8559-4e31-908f-a2e71f400722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:40.647110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5c993769-3cbc-46f4-bf07-4a98ba8e1301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94acbacaac6a4ba4a3211c8d885c6e69', 'request_id': 'req-5c993769-3cbc-46f4-bf07-4a98ba8e1301'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:40.660745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5c993769-3cbc-46f4-bf07-4a98ba8e1301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94acbacaac6a4ba4a3211c8d885c6e69', 'request_id': 'req-5c993769-3cbc-46f4-bf07-4a98ba8e1301'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:40.678643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5c993769-3cbc-46f4-bf07-4a98ba8e1301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94acbacaac6a4ba4a3211c8d885c6e69', 'request_id': 'req-5c993769-3cbc-46f4-bf07-4a98ba8e1301'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:40.707770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5c993769-3cbc-46f4-bf07-4a98ba8e1301 admin admin] The token cache is being invalidate because project 94acbacaac6a4ba4a3211c8d885c6e69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:40.709402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5c993769-3cbc-46f4-bf07-4a98ba8e1301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:40.710655 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5c993769-3cbc-46f4-bf07-4a98ba8e1301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:40.716737 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1030/2088] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:40.722408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1058/2088] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:40 2022] DELETE /identity/v3/projects/94acbacaac6a4ba4a3211c8d885c6e69 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:40.723333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6b4f5-b209-4905-987c-e9e72b958862 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:40.724199 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6b4f5-b209-4905-987c-e9e72b958862 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:40.724777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6b4f5-b209-4905-987c-e9e72b958862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:40.759261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-b3a6b4f5-b209-4905-987c-e9e72b958862 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:40.761464 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b3a6b4f5-b209-4905-987c-e9e72b958862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:40.874948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:40.960773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1059/2089] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:40.972911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3a14a-13f6-4344-8f31-cae71fbacfb2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:40.978222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3a14a-13f6-4344-8f31-cae71fbacfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:40.978703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3a14a-13f6-4344-8f31-cae71fbacfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:41.041277 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-38a3a14a-13f6-4344-8f31-cae71fbacfb2 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:41.043613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-38a3a14a-13f6-4344-8f31-cae71fbacfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:41.066091 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1031/2090] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:41.070190 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6b4f5-b209-4905-987c-e9e72b958862 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:41.096638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6b4f5-b209-4905-987c-e9e72b958862 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:41.121382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6b5ee-107c-4502-9f25-894df1953440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:41.124205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6b5ee-107c-4502-9f25-894df1953440 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:41.124502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6b5ee-107c-4502-9f25-894df1953440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:41.124785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6b5ee-107c-4502-9f25-894df1953440 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:41.129365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6b5ee-107c-4502-9f25-894df1953440 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:37:41.129700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6b5ee-107c-4502-9f25-894df1953440 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:41.135188 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6b5ee-107c-4502-9f25-894df1953440 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:41.169917 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1060/2091] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:37:41.171424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3a14a-13f6-4344-8f31-cae71fbacfb2 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:41.196585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3a14a-13f6-4344-8f31-cae71fbacfb2 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:41.203212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec394e-af48-489f-8567-9909395aabac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:41.205029 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec394e-af48-489f-8567-9909395aabac admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:41.206540 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec394e-af48-489f-8567-9909395aabac admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:41.206925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec394e-af48-489f-8567-9909395aabac admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:41.207785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec394e-af48-489f-8567-9909395aabac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:41.211511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec394e-af48-489f-8567-9909395aabac admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:41.297649 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1032/2092] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:41 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:41.302657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6b5ee-107c-4502-9f25-894df1953440 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:41.322797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6b5ee-107c-4502-9f25-894df1953440 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:41.439617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba0e5d-07f1-4a94-a808-76806aa1f600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:41.440300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba0e5d-07f1-4a94-a808-76806aa1f600 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:41.440606 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba0e5d-07f1-4a94-a808-76806aa1f600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:41.440870 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba0e5d-07f1-4a94-a808-76806aa1f600 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:41.441643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba0e5d-07f1-4a94-a808-76806aa1f600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:41.443993 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba0e5d-07f1-4a94-a808-76806aa1f600 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:41.480298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1061/2093] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:41 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:41.484281 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec394e-af48-489f-8567-9909395aabac admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:41.501909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec394e-af48-489f-8567-9909395aabac admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:41.510282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9eb23a-57df-4994-880e-0f3a8fc84b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:41.511386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9eb23a-57df-4994-880e-0f3a8fc84b82 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:41.511673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9eb23a-57df-4994-880e-0f3a8fc84b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:41.511951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9eb23a-57df-4994-880e-0f3a8fc84b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:41.512812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9eb23a-57df-4994-880e-0f3a8fc84b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:41.514700 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9eb23a-57df-4994-880e-0f3a8fc84b82 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:41.541414 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1033/2094] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:41.545390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba0e5d-07f1-4a94-a808-76806aa1f600 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:41.560040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba0e5d-07f1-4a94-a808-76806aa1f600 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:41.565305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a071d4f-7ba5-472f-9926-3f56215696c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:41.565994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a071d4f-7ba5-472f-9926-3f56215696c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:41.566293 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a071d4f-7ba5-472f-9926-3f56215696c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:41.566562 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a071d4f-7ba5-472f-9926-3f56215696c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:41.567320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a071d4f-7ba5-472f-9926-3f56215696c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:41.569209 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a071d4f-7ba5-472f-9926-3f56215696c0 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:41.581605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1062/2095] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:41.586126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9eb23a-57df-4994-880e-0f3a8fc84b82 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:41.597574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9eb23a-57df-4994-880e-0f3a8fc84b82 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:41.607979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e7e87-d07e-4dbc-a688-d218a5676de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:41.609138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e7e87-d07e-4dbc-a688-d218a5676de3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:41.609435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e7e87-d07e-4dbc-a688-d218a5676de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:41.609762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e7e87-d07e-4dbc-a688-d218a5676de3 admin admin] PATH_INFO: `/v3/projects/d26a3512c9bb48a6825b957ef7dcab41/users/5be2e3868fec4bcfbd15254bf94175ec/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:41.647561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e7e87-d07e-4dbc-a688-d218a5676de3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d26a3512c9bb48a6825b957ef7dcab41, user_id=5be2e3868fec4bcfbd15254bf94175ec, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:41.657107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e7e87-d07e-4dbc-a688-d218a5676de3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:41.688207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1034/2096] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:41 2022] PUT /identity/v3/projects/d26a3512c9bb48a6825b957ef7dcab41/users/5be2e3868fec4bcfbd15254bf94175ec/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:41.693640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37dcda-9972-4225-9217-c897e827898a None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:41.694439 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37dcda-9972-4225-9217-c897e827898a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:41.694927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37dcda-9972-4225-9217-c897e827898a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:41.759506 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-5f37dcda-9972-4225-9217-c897e827898a None None] MFA Rules not processed for user `5be2e3868fec4bcfbd15254bf94175ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:41.761366 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5f37dcda-9972-4225-9217-c897e827898a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:41.903167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1063/2097] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:41.903430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e7e87-d07e-4dbc-a688-d218a5676de3 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:42.042092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e7e87-d07e-4dbc-a688-d218a5676de3 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:42.047483 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a65d3-ee5a-42ba-9f6a-251999edade5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:42.048205 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a65d3-ee5a-42ba-9f6a-251999edade5 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:42.049256 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a65d3-ee5a-42ba-9f6a-251999edade5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:42.049256 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a65d3-ee5a-42ba-9f6a-251999edade5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:42.057521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a65d3-ee5a-42ba-9f6a-251999edade5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:42.057521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a65d3-ee5a-42ba-9f6a-251999edade5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:42.057521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:42.090246 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1035/2098] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:41 2022] POST /identity/v3/projects => generated 369 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:37:42.094437 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37dcda-9972-4225-9217-c897e827898a None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:42.109234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37dcda-9972-4225-9217-c897e827898a None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:42.198586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:42.257244 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07aedc8-5200-483e-a009-5030da8eb5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:42.257244 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07aedc8-5200-483e-a009-5030da8eb5f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:42.257244 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07aedc8-5200-483e-a009-5030da8eb5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:42.257662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07aedc8-5200-483e-a009-5030da8eb5f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:42.258230 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07aedc8-5200-483e-a009-5030da8eb5f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:42.260375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07aedc8-5200-483e-a009-5030da8eb5f4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:42.288476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1064/2099] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:42 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:37:42.292673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a65d3-ee5a-42ba-9f6a-251999edade5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:42.305303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a65d3-ee5a-42ba-9f6a-251999edade5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:42.311358 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15f159-2283-4a10-8bd6-fd264b3d5149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:42.312168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15f159-2283-4a10-8bd6-fd264b3d5149 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:42.312457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15f159-2283-4a10-8bd6-fd264b3d5149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:42.312736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15f159-2283-4a10-8bd6-fd264b3d5149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:42.313560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15f159-2283-4a10-8bd6-fd264b3d5149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:42.315420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15f159-2283-4a10-8bd6-fd264b3d5149 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:42.330797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1036/2100] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:42.336777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07aedc8-5200-483e-a009-5030da8eb5f4 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:42.348824 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07aedc8-5200-483e-a009-5030da8eb5f4 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:42.363989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e430153-57a0-4348-b7a2-807407b2d216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:42.368930 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e430153-57a0-4348-b7a2-807407b2d216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:42.369465 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e430153-57a0-4348-b7a2-807407b2d216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:42.369848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e430153-57a0-4348-b7a2-807407b2d216 admin admin] PATH_INFO: `/v3/projects/a3e20c9ba03c45ba834c492f66a17739/users/10843bd3d33d48dfb7435f9c3abe5a1b/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:42.395329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e430153-57a0-4348-b7a2-807407b2d216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e20c9ba03c45ba834c492f66a17739, user_id=10843bd3d33d48dfb7435f9c3abe5a1b, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:42.398724 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e430153-57a0-4348-b7a2-807407b2d216 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:42.418416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1065/2101] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:42 2022] PUT /identity/v3/projects/a3e20c9ba03c45ba834c492f66a17739/users/10843bd3d33d48dfb7435f9c3abe5a1b/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:42.422605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15f159-2283-4a10-8bd6-fd264b3d5149 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:42.440084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15f159-2283-4a10-8bd6-fd264b3d5149 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:42.584372 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009739b-de0e-401d-9449-cf61271f18cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:42.585664 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009739b-de0e-401d-9449-cf61271f18cf admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:42.586213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009739b-de0e-401d-9449-cf61271f18cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:42.587244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009739b-de0e-401d-9449-cf61271f18cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:42.588019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009739b-de0e-401d-9449-cf61271f18cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:42.589722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009739b-de0e-401d-9449-cf61271f18cf admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:42.609382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1037/2102] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:42.613803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e430153-57a0-4348-b7a2-807407b2d216 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:42.637960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e430153-57a0-4348-b7a2-807407b2d216 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:42.646603 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8dc751-ce5e-4b8e-bd5d-f4c468017bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:42.648948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dc751-ce5e-4b8e-bd5d-f4c468017bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:42.649736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dc751-ce5e-4b8e-bd5d-f4c468017bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:42.650408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dc751-ce5e-4b8e-bd5d-f4c468017bc9 admin admin] PATH_INFO: `/v3/projects/a3e20c9ba03c45ba834c492f66a17739/users/10843bd3d33d48dfb7435f9c3abe5a1b/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:42.654483 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8dc751-ce5e-4b8e-bd5d-f4c468017bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e20c9ba03c45ba834c492f66a17739, user_id=10843bd3d33d48dfb7435f9c3abe5a1b, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:42.660544 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8dc751-ce5e-4b8e-bd5d-f4c468017bc9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:42.689155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1066/2103] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:42 2022] PUT /identity/v3/projects/a3e20c9ba03c45ba834c492f66a17739/users/10843bd3d33d48dfb7435f9c3abe5a1b/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:42.704310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812a88e-6f8a-4e92-84bd-03dd868551e6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:42.705171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812a88e-6f8a-4e92-84bd-03dd868551e6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:42.705385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812a88e-6f8a-4e92-84bd-03dd868551e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:42.764861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4812a88e-6f8a-4e92-84bd-03dd868551e6 None None] MFA Rules not processed for user `10843bd3d33d48dfb7435f9c3abe5a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:42.767592 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4812a88e-6f8a-4e92-84bd-03dd868551e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:42.955982 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1038/2104] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:42 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:42.970505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8dc751-ce5e-4b8e-bd5d-f4c468017bc9 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:42.977712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-2e8dc751-ce5e-4b8e-bd5d-f4c468017bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:43.147381 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8dc751-ce5e-4b8e-bd5d-f4c468017bc9 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:43.151447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d0052-3628-400e-a863-fdd3f6f8f923 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:43.152138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d0052-3628-400e-a863-fdd3f6f8f923 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:43.152397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d0052-3628-400e-a863-fdd3f6f8f923 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:43.152682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d0052-3628-400e-a863-fdd3f6f8f923 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:43.169194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d0052-3628-400e-a863-fdd3f6f8f923 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:43.170820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d0052-3628-400e-a863-fdd3f6f8f923 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:43.183271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1067/2105] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:42 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:45.726106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812a88e-6f8a-4e92-84bd-03dd868551e6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:45.959087 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:46.088724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812a88e-6f8a-4e92-84bd-03dd868551e6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:46.120354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:46.122514 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:46.123285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:46.123962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] PATH_INFO: `/v3/users/5be2e3868fec4bcfbd15254bf94175ec` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:46.126485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5be2e3868fec4bcfbd15254bf94175ec)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:46.131567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:46.271999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5be2e3868fec4bcfbd15254bf94175ec'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:46.273791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5be2e3868fec4bcfbd15254bf94175ec'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:46.294134 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5be2e3868fec4bcfbd15254bf94175ec'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:46.296506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5be2e3868fec4bcfbd15254bf94175ec'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:46.328477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5be2e3868fec4bcfbd15254bf94175ec'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:46.367858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1039/2106] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:45 2022] DELETE /identity/v3/users/5be2e3868fec4bcfbd15254bf94175ec => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:46.374512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334d2a8-ba29-4db2-823d-4c47075e42d4 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:46.375402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334d2a8-ba29-4db2-823d-4c47075e42d4 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:46.375402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334d2a8-ba29-4db2-823d-4c47075e42d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:46.415132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-2334d2a8-ba29-4db2-823d-4c47075e42d4 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:46.416781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-2334d2a8-ba29-4db2-823d-4c47075e42d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:46.512884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1068/2107] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:46.527634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:46.530074 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:46.626416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f3c4a-9442-4182-a905-fedfdfb5f829 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:46.627550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f3c4a-9442-4182-a905-fedfdfb5f829 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:46.628026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f3c4a-9442-4182-a905-fedfdfb5f829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:46.675143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-a77f3c4a-9442-4182-a905-fedfdfb5f829 None None] MFA Rules not processed for user `ffa54c295b764c07ba5d4c1d911fe7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:46.678093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a77f3c4a-9442-4182-a905-fedfdfb5f829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:46.687511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:46.736151 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ed637-25a7-4e9f-ab9a-2d5994578c3e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:46.745507 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c329301-cb76-4d13-99d0-8cbb76d3b8df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:46.747458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c329301-cb76-4d13-99d0-8cbb76d3b8df service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:46.748564 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c329301-cb76-4d13-99d0-8cbb76d3b8df service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:46.749393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c329301-cb76-4d13-99d0-8cbb76d3b8df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:46.768012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c329301-cb76-4d13-99d0-8cbb76d3b8df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:46.769565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c329301-cb76-4d13-99d0-8cbb76d3b8df service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:46.790847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1040/2108] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:37:46.819872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1069/2109] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:47.205023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c329301-cb76-4d13-99d0-8cbb76d3b8df service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:47.209023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5c329301-cb76-4d13-99d0-8cbb76d3b8df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:47.225008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c329301-cb76-4d13-99d0-8cbb76d3b8df service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:47.230640 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:47.231868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:47.232152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:47.232439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] PATH_INFO: `/v3/projects/d26a3512c9bb48a6825b957ef7dcab41` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:47.234087 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d26a3512c9bb48a6825b957ef7dcab41)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:47.236756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:47.339941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:47.381184 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd26a3512c9bb48a6825b957ef7dcab41', 'request_id': 'req-a622732e-734e-4b50-9bc3-ef54da19c94c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.392292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd26a3512c9bb48a6825b957ef7dcab41', 'request_id': 'req-a622732e-734e-4b50-9bc3-ef54da19c94c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.406908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd26a3512c9bb48a6825b957ef7dcab41', 'request_id': 'req-a622732e-734e-4b50-9bc3-ef54da19c94c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.428675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f3c4a-9442-4182-a905-fedfdfb5f829 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:47.432669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a77f3c4a-9442-4182-a905-fedfdfb5f829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:47.444721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] The token cache is being invalidate because project d26a3512c9bb48a6825b957ef7dcab41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:47.445257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.446044 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.452196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1041/2110] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:47 2022] DELETE /identity/v3/projects/d26a3512c9bb48a6825b957ef7dcab41 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:47.453670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:47.457532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:47.467779 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f3c4a-9442-4182-a905-fedfdfb5f829 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:47.574541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:47.575458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:47.575811 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:47.576294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] PATH_INFO: `/v3/users/0b7cde3d137a44d1bb2ce419f973f61a` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:47.579947 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b7cde3d137a44d1bb2ce419f973f61a)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:47.581990 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:47.585023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622732e-734e-4b50-9bc3-ef54da19c94c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:47.592155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458ef56-7e2b-420b-90a4-710f816d65e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:47.593378 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458ef56-7e2b-420b-90a4-710f816d65e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:47.593850 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458ef56-7e2b-420b-90a4-710f816d65e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:47.594274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458ef56-7e2b-420b-90a4-710f816d65e1 admin admin] PATH_INFO: `/v3/users/10843bd3d33d48dfb7435f9c3abe5a1b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:47.596340 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458ef56-7e2b-420b-90a4-710f816d65e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10843bd3d33d48dfb7435f9c3abe5a1b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:47.598601 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458ef56-7e2b-420b-90a4-710f816d65e1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:47.672467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2458ef56-7e2b-420b-90a4-710f816d65e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10843bd3d33d48dfb7435f9c3abe5a1b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.674553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2458ef56-7e2b-420b-90a4-710f816d65e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10843bd3d33d48dfb7435f9c3abe5a1b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.688904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2458ef56-7e2b-420b-90a4-710f816d65e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10843bd3d33d48dfb7435f9c3abe5a1b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.690684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2458ef56-7e2b-420b-90a4-710f816d65e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10843bd3d33d48dfb7435f9c3abe5a1b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.697200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b7cde3d137a44d1bb2ce419f973f61a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.700023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b7cde3d137a44d1bb2ce419f973f61a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.713456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b7cde3d137a44d1bb2ce419f973f61a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.718127 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2458ef56-7e2b-420b-90a4-710f816d65e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10843bd3d33d48dfb7435f9c3abe5a1b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.738768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b7cde3d137a44d1bb2ce419f973f61a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.739813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b7cde3d137a44d1bb2ce419f973f61a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:47.758228 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1042/2111] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:47 2022] DELETE /identity/v3/users/10843bd3d33d48dfb7435f9c3abe5a1b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:47.774803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1070/2112] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:47 2022] DELETE /identity/v3/users/0b7cde3d137a44d1bb2ce419f973f61a => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:47.778449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2d538-db56-4bba-9823-2c9c7a86e50d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:47.779059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2d538-db56-4bba-9823-2c9c7a86e50d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:47.779646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2d538-db56-4bba-9823-2c9c7a86e50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:47.813368 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-49f2d538-db56-4bba-9823-2c9c7a86e50d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:47.815549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-49f2d538-db56-4bba-9823-2c9c7a86e50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:47.891133 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:47.930291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1043/2113] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:47.953555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:47.960132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:48.042818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:48.092124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522173e-e726-4d9e-a53e-ad73721047a5 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:48.098410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100ed40-9f47-4d66-9bc4-fba80f8c9480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:48.100655 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100ed40-9f47-4d66-9bc4-fba80f8c9480 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:48.101383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100ed40-9f47-4d66-9bc4-fba80f8c9480 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:48.102066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100ed40-9f47-4d66-9bc4-fba80f8c9480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:48.118192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100ed40-9f47-4d66-9bc4-fba80f8c9480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:48.120085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100ed40-9f47-4d66-9bc4-fba80f8c9480 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:48.131366 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1071/2114] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:48.158502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2d538-db56-4bba-9823-2c9c7a86e50d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:48.160007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-49f2d538-db56-4bba-9823-2c9c7a86e50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:48.173455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2d538-db56-4bba-9823-2c9c7a86e50d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:48.193139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f89fd-5025-4050-ae6a-183bb279da03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:48.194934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f89fd-5025-4050-ae6a-183bb279da03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:48.194934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f89fd-5025-4050-ae6a-183bb279da03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:48.195727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f89fd-5025-4050-ae6a-183bb279da03 admin admin] PATH_INFO: `/v3/projects/a3e20c9ba03c45ba834c492f66a17739` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:48.198925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f89fd-5025-4050-ae6a-183bb279da03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e20c9ba03c45ba834c492f66a17739)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:48.201799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f89fd-5025-4050-ae6a-183bb279da03 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:48.324092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4d7f89fd-5025-4050-ae6a-183bb279da03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e20c9ba03c45ba834c492f66a17739', 'request_id': 'req-4d7f89fd-5025-4050-ae6a-183bb279da03'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:48.338731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4d7f89fd-5025-4050-ae6a-183bb279da03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e20c9ba03c45ba834c492f66a17739', 'request_id': 'req-4d7f89fd-5025-4050-ae6a-183bb279da03'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:48.363854 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4d7f89fd-5025-4050-ae6a-183bb279da03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e20c9ba03c45ba834c492f66a17739', 'request_id': 'req-4d7f89fd-5025-4050-ae6a-183bb279da03'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:48.391494 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4d7f89fd-5025-4050-ae6a-183bb279da03 admin admin] The token cache is being invalidate because project a3e20c9ba03c45ba834c492f66a17739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:48.393186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4d7f89fd-5025-4050-ae6a-183bb279da03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:48.396666 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4d7f89fd-5025-4050-ae6a-183bb279da03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:48.399497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1044/2115] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:48 2022] DELETE /identity/v3/projects/a3e20c9ba03c45ba834c492f66a17739 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:48.411583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33256ae4-8321-4008-b777-4139c35d366e None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:48.418048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33256ae4-8321-4008-b777-4139c35d366e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:48.418613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33256ae4-8321-4008-b777-4139c35d366e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:48.566527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-33256ae4-8321-4008-b777-4139c35d366e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:48.568710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-33256ae4-8321-4008-b777-4139c35d366e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:48.701449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1072/2116] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:48.708894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2f6d9-237d-45f0-9da9-c5ea3aa0ba47 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:48.709297 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2f6d9-237d-45f0-9da9-c5ea3aa0ba47 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:48.709701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2f6d9-237d-45f0-9da9-c5ea3aa0ba47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:48.737248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b1e2f6d9-237d-45f0-9da9-c5ea3aa0ba47 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:48.739306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b1e2f6d9-237d-45f0-9da9-c5ea3aa0ba47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:48.753460 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1045/2117] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:48.757942 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33256ae4-8321-4008-b777-4139c35d366e None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:48.805741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33256ae4-8321-4008-b777-4139c35d366e None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:48.837385 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac69bec-2d76-4d20-a77a-279c902b6787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:48.840214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac69bec-2d76-4d20-a77a-279c902b6787 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:48.841172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac69bec-2d76-4d20-a77a-279c902b6787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:48.841566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac69bec-2d76-4d20-a77a-279c902b6787 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:48.842703 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac69bec-2d76-4d20-a77a-279c902b6787 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:37:48.843133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac69bec-2d76-4d20-a77a-279c902b6787 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:48.846022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac69bec-2d76-4d20-a77a-279c902b6787 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:48.883528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1073/2118] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:37:48.889179 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2f6d9-237d-45f0-9da9-c5ea3aa0ba47 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:48.912948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2f6d9-237d-45f0-9da9-c5ea3aa0ba47 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:48.923500 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d1c08-03c8-4fe4-8047-d10d33f5e61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:48.924974 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d1c08-03c8-4fe4-8047-d10d33f5e61d admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:48.925584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d1c08-03c8-4fe4-8047-d10d33f5e61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:48.925929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d1c08-03c8-4fe4-8047-d10d33f5e61d admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:48.926722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d1c08-03c8-4fe4-8047-d10d33f5e61d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:48.929766 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d1c08-03c8-4fe4-8047-d10d33f5e61d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.007821 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1046/2119] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:48 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:49.011961 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac69bec-2d76-4d20-a77a-279c902b6787 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.035468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac69bec-2d76-4d20-a77a-279c902b6787 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.120546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d1c08-03c8-4fe4-8047-d10d33f5e61d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.122780 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3d5d1c08-03c8-4fe4-8047-d10d33f5e61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:49.126499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f6931-eba0-405c-a155-9cbb5b0d6cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.127354 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f6931-eba0-405c-a155-9cbb5b0d6cdb admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.127766 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f6931-eba0-405c-a155-9cbb5b0d6cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.128144 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f6931-eba0-405c-a155-9cbb5b0d6cdb admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.128997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f6931-eba0-405c-a155-9cbb5b0d6cdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.133149 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f6931-eba0-405c-a155-9cbb5b0d6cdb admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.139286 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d1c08-03c8-4fe4-8047-d10d33f5e61d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.147376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaf226-bc46-49cc-814d-cdbc38edf5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.148571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaf226-bc46-49cc-814d-cdbc38edf5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.149464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaf226-bc46-49cc-814d-cdbc38edf5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.150690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaf226-bc46-49cc-814d-cdbc38edf5a7 admin admin] PATH_INFO: `/v3/projects/1547556b58444fdebb8ec8641e424509` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.154124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabaf226-bc46-49cc-814d-cdbc38edf5a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1547556b58444fdebb8ec8641e424509)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.160407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabaf226-bc46-49cc-814d-cdbc38edf5a7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.185117 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1074/2120] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:49 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:37:49.190259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f6931-eba0-405c-a155-9cbb5b0d6cdb admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.210568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f6931-eba0-405c-a155-9cbb5b0d6cdb admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.222661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7df18-3974-4a2e-bd0c-ee9ea20f1c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.224807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7df18-3974-4a2e-bd0c-ee9ea20f1c67 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.225750 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7df18-3974-4a2e-bd0c-ee9ea20f1c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.226848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7df18-3974-4a2e-bd0c-ee9ea20f1c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.228034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7df18-3974-4a2e-bd0c-ee9ea20f1c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.230761 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7df18-3974-4a2e-bd0c-ee9ea20f1c67 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.261734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1075/2121] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:37:49.262991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7df18-3974-4a2e-bd0c-ee9ea20f1c67 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.277952 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7df18-3974-4a2e-bd0c-ee9ea20f1c67 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.279023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cabaf226-bc46-49cc-814d-cdbc38edf5a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1547556b58444fdebb8ec8641e424509', 'request_id': 'req-cabaf226-bc46-49cc-814d-cdbc38edf5a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:49.290214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cabaf226-bc46-49cc-814d-cdbc38edf5a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1547556b58444fdebb8ec8641e424509', 'request_id': 'req-cabaf226-bc46-49cc-814d-cdbc38edf5a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:49.317222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cabaf226-bc46-49cc-814d-cdbc38edf5a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1547556b58444fdebb8ec8641e424509', 'request_id': 'req-cabaf226-bc46-49cc-814d-cdbc38edf5a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:49.330542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:49.343703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cabaf226-bc46-49cc-814d-cdbc38edf5a7 admin admin] The token cache is being invalidate because project 1547556b58444fdebb8ec8641e424509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:49.343899 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cabaf226-bc46-49cc-814d-cdbc38edf5a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:49.345040 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cabaf226-bc46-49cc-814d-cdbc38edf5a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:49.351768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1047/2122] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:49 2022] DELETE /identity/v3/projects/1547556b58444fdebb8ec8641e424509 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:49.360449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e61303-4355-44d6-8e55-189de39904cb None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.360449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e61303-4355-44d6-8e55-189de39904cb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.360449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e61303-4355-44d6-8e55-189de39904cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.396898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b5e61303-4355-44d6-8e55-189de39904cb None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:49.398206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b5e61303-4355-44d6-8e55-189de39904cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:49.407196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f763fd-ccad-49c2-9048-a4ae3f2bb9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.408383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f763fd-ccad-49c2-9048-a4ae3f2bb9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.408902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f763fd-ccad-49c2-9048-a4ae3f2bb9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.409356 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f763fd-ccad-49c2-9048-a4ae3f2bb9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.410876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f763fd-ccad-49c2-9048-a4ae3f2bb9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.414221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:49.415326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f763fd-ccad-49c2-9048-a4ae3f2bb9a5 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.434041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1076/2123] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:49.438582 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f763fd-ccad-49c2-9048-a4ae3f2bb9a5 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.440675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-17f763fd-ccad-49c2-9048-a4ae3f2bb9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:49.457914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f763fd-ccad-49c2-9048-a4ae3f2bb9a5 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.465281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e4bb7-851a-49dd-b8fa-eaa4545e0990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.466196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e4bb7-851a-49dd-b8fa-eaa4545e0990 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.466482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e4bb7-851a-49dd-b8fa-eaa4545e0990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.466762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e4bb7-851a-49dd-b8fa-eaa4545e0990 admin admin] PATH_INFO: `/v3/projects/5388981bda864839935d89cee9939a6c/users/dffe21f69a7c4ed7a20a0f206d40e27f/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.486328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e4bb7-851a-49dd-b8fa-eaa4545e0990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5388981bda864839935d89cee9939a6c, user_id=dffe21f69a7c4ed7a20a0f206d40e27f, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.491045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e4bb7-851a-49dd-b8fa-eaa4545e0990 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.495351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1048/2124] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:49.496892 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f7ddf-aed6-4cc0-82c9-e3f274a957ff None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.497237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f7ddf-aed6-4cc0-82c9-e3f274a957ff None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.497526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f7ddf-aed6-4cc0-82c9-e3f274a957ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.515967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1077/2125] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:49 2022] PUT /identity/v3/projects/5388981bda864839935d89cee9939a6c/users/dffe21f69a7c4ed7a20a0f206d40e27f/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:49.526309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f52f7ddf-aed6-4cc0-82c9-e3f274a957ff None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:49.527864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030c431-db4a-495a-948a-da2f2ae167a8 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.528145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030c431-db4a-495a-948a-da2f2ae167a8 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.528424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030c431-db4a-495a-948a-da2f2ae167a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.530712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f52f7ddf-aed6-4cc0-82c9-e3f274a957ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:49.566348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-a030c431-db4a-495a-948a-da2f2ae167a8 None None] MFA Rules not processed for user `dffe21f69a7c4ed7a20a0f206d40e27f`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:49.567875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a030c431-db4a-495a-948a-da2f2ae167a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:49.620310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1049/2126] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:49.624745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f7ddf-aed6-4cc0-82c9-e3f274a957ff None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.638536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f7ddf-aed6-4cc0-82c9-e3f274a957ff None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.647064 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d85faa-0397-4701-9661-4820dc4681a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.648093 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d85faa-0397-4701-9661-4820dc4681a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.648621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d85faa-0397-4701-9661-4820dc4681a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.648762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d85faa-0397-4701-9661-4820dc4681a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.649735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d85faa-0397-4701-9661-4820dc4681a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:37:49.650047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d85faa-0397-4701-9661-4820dc4681a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.651883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d85faa-0397-4701-9661-4820dc4681a5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.671193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1078/2127] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:49.671745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1050/2128] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:37:49.673184 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d85faa-0397-4701-9661-4820dc4681a5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.673300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030c431-db4a-495a-948a-da2f2ae167a8 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.685162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030c431-db4a-495a-948a-da2f2ae167a8 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.686115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d85faa-0397-4701-9661-4820dc4681a5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.690118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538555f-3b6d-49d8-9b2d-6499b02c18c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.690865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538555f-3b6d-49d8-9b2d-6499b02c18c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.691561 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538555f-3b6d-49d8-9b2d-6499b02c18c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.691561 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538555f-3b6d-49d8-9b2d-6499b02c18c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.692286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538555f-3b6d-49d8-9b2d-6499b02c18c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.692651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe75355-24da-4a44-9a7f-1a56c7b427b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.693441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe75355-24da-4a44-9a7f-1a56c7b427b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.693871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe75355-24da-4a44-9a7f-1a56c7b427b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.694246 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe75355-24da-4a44-9a7f-1a56c7b427b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.694633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538555f-3b6d-49d8-9b2d-6499b02c18c7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.695086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe75355-24da-4a44-9a7f-1a56c7b427b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.697312 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe75355-24da-4a44-9a7f-1a56c7b427b9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.726604 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1079/2129] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:49 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:49.732419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1051/2130] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:49 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:49.732580 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538555f-3b6d-49d8-9b2d-6499b02c18c7 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.734057 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe75355-24da-4a44-9a7f-1a56c7b427b9 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.801988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538555f-3b6d-49d8-9b2d-6499b02c18c7 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.803661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe75355-24da-4a44-9a7f-1a56c7b427b9 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.808220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fabf4a-4a2e-4b5f-a863-b65934eee8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.808913 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fabf4a-4a2e-4b5f-a863-b65934eee8bb admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.809077 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257d49f-04ef-478e-a326-b632dc9dff71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.809278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fabf4a-4a2e-4b5f-a863-b65934eee8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.809496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fabf4a-4a2e-4b5f-a863-b65934eee8bb admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.810193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fabf4a-4a2e-4b5f-a863-b65934eee8bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.810302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257d49f-04ef-478e-a326-b632dc9dff71 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.810561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257d49f-04ef-478e-a326-b632dc9dff71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.810847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257d49f-04ef-478e-a326-b632dc9dff71 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.811538 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0257d49f-04ef-478e-a326-b632dc9dff71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.812141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fabf4a-4a2e-4b5f-a863-b65934eee8bb admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.813559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0257d49f-04ef-478e-a326-b632dc9dff71 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.823037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:49.838211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1052/2131] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:49 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:37:49.838909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1080/2132] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:49 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:37:49.842387 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257d49f-04ef-478e-a326-b632dc9dff71 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.842631 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fabf4a-4a2e-4b5f-a863-b65934eee8bb admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.853316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257d49f-04ef-478e-a326-b632dc9dff71 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.853520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fabf4a-4a2e-4b5f-a863-b65934eee8bb admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.858203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93917c74-ef4b-4fbf-8e4a-f89aa7771746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.858496 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30220a-10b4-44d1-9be1-5e0480596b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.858911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93917c74-ef4b-4fbf-8e4a-f89aa7771746 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.859203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93917c74-ef4b-4fbf-8e4a-f89aa7771746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.859484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93917c74-ef4b-4fbf-8e4a-f89aa7771746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.859627 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30220a-10b4-44d1-9be1-5e0480596b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.860033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30220a-10b4-44d1-9be1-5e0480596b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.860219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93917c74-ef4b-4fbf-8e4a-f89aa7771746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.860489 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30220a-10b4-44d1-9be1-5e0480596b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.861353 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30220a-10b4-44d1-9be1-5e0480596b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.861873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93917c74-ef4b-4fbf-8e4a-f89aa7771746 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.863414 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30220a-10b4-44d1-9be1-5e0480596b72 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.873950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1081/2133] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:37:49.877935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93917c74-ef4b-4fbf-8e4a-f89aa7771746 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.879024 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1053/2134] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:37:49.886536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30220a-10b4-44d1-9be1-5e0480596b72 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.890881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93917c74-ef4b-4fbf-8e4a-f89aa7771746 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.901966 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c36c07-79c6-470d-a1b9-fd7135eccc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.902786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c36c07-79c6-470d-a1b9-fd7135eccc72 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.903080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c36c07-79c6-470d-a1b9-fd7135eccc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.903363 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c36c07-79c6-470d-a1b9-fd7135eccc72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.903875 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:49.904423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c36c07-79c6-470d-a1b9-fd7135eccc72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.906265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c36c07-79c6-470d-a1b9-fd7135eccc72 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.906846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30220a-10b4-44d1-9be1-5e0480596b72 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.915616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7b595-9869-4080-826c-d3632fc88684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.916764 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7b595-9869-4080-826c-d3632fc88684 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.917312 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7b595-9869-4080-826c-d3632fc88684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.917815 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7b595-9869-4080-826c-d3632fc88684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.919181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7b595-9869-4080-826c-d3632fc88684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.920051 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1082/2135] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:37:49.922086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7b595-9869-4080-826c-d3632fc88684 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.924557 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c36c07-79c6-470d-a1b9-fd7135eccc72 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.934706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1054/2136] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:37:49.936399 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c36c07-79c6-470d-a1b9-fd7135eccc72 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.938899 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7b595-9869-4080-826c-d3632fc88684 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:49.941297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746240c-1d4e-4dae-8d84-c3936bb58a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.942195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746240c-1d4e-4dae-8d84-c3936bb58a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.942492 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746240c-1d4e-4dae-8d84-c3936bb58a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.943345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746240c-1d4e-4dae-8d84-c3936bb58a0d admin admin] PATH_INFO: `/v3/projects/e1ea6b05b673485f8c00c4f175878099/users/5b68ff92655141df92b3470ffc6a493b/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.949736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7b595-9869-4080-826c-d3632fc88684 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:49.954715 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec498e-7296-45b2-9c42-0d6f1aab0811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:49.955621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec498e-7296-45b2-9c42-0d6f1aab0811 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.955907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec498e-7296-45b2-9c42-0d6f1aab0811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.956191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec498e-7296-45b2-9c42-0d6f1aab0811 admin admin] PATH_INFO: `/v3/projects/61a30a10fad14c618aa6a4233dabecf4/users/9bd46764b8cf4a1aae58f8ba4b813920/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:49.960170 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746240c-1d4e-4dae-8d84-c3936bb58a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ea6b05b673485f8c00c4f175878099, user_id=5b68ff92655141df92b3470ffc6a493b, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.962963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746240c-1d4e-4dae-8d84-c3936bb58a0d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.977920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec498e-7296-45b2-9c42-0d6f1aab0811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a30a10fad14c618aa6a4233dabecf4, user_id=9bd46764b8cf4a1aae58f8ba4b813920, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:49.979722 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1083/2137] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:49 2022] PUT /identity/v3/projects/e1ea6b05b673485f8c00c4f175878099/users/5b68ff92655141df92b3470ffc6a493b/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:49.990141 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec498e-7296-45b2-9c42-0d6f1aab0811 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:49.996524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f2816-0482-4791-a10f-01fec2640140 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:49.997137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f2816-0482-4791-a10f-01fec2640140 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:49.997547 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f2816-0482-4791-a10f-01fec2640140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:50.007870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1055/2138] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:49 2022] PUT /identity/v3/projects/61a30a10fad14c618aa6a4233dabecf4/users/9bd46764b8cf4a1aae58f8ba4b813920/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:50.025302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d262a7-eefc-4beb-bec4-05fbeb98b704 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:50.025598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d262a7-eefc-4beb-bec4-05fbeb98b704 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:50.025879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d262a7-eefc-4beb-bec4-05fbeb98b704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:50.035978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d76f2816-0482-4791-a10f-01fec2640140 None None] MFA Rules not processed for user `5b68ff92655141df92b3470ffc6a493b`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:50.038330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d76f2816-0482-4791-a10f-01fec2640140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:50.058897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c1d262a7-eefc-4beb-bec4-05fbeb98b704 None None] MFA Rules not processed for user `9bd46764b8cf4a1aae58f8ba4b813920`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:50.060214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c1d262a7-eefc-4beb-bec4-05fbeb98b704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:50.135657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1084/2139] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:50.148976 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f2816-0482-4791-a10f-01fec2640140 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:50.150085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d76f2816-0482-4791-a10f-01fec2640140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:50.172596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1056/2140] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:50.178026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d262a7-eefc-4beb-bec4-05fbeb98b704 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:50.222035 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f2816-0482-4791-a10f-01fec2640140 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:50.232733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d994192-e53f-459f-b1e4-7b9e8b2de39d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:50.233546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d994192-e53f-459f-b1e4-7b9e8b2de39d service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:50.233827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d994192-e53f-459f-b1e4-7b9e8b2de39d service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:50.234099 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d994192-e53f-459f-b1e4-7b9e8b2de39d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:50.251775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d994192-e53f-459f-b1e4-7b9e8b2de39d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:50.254212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d994192-e53f-459f-b1e4-7b9e8b2de39d service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:50.278641 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1085/2141] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:50.295003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d262a7-eefc-4beb-bec4-05fbeb98b704 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:50.305890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9dcb9-4df0-46fb-bec4-8864a74e0c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:50.307324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9dcb9-4df0-46fb-bec4-8864a74e0c74 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:50.308125 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9dcb9-4df0-46fb-bec4-8864a74e0c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:50.308595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9dcb9-4df0-46fb-bec4-8864a74e0c74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:50.311227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9dcb9-4df0-46fb-bec4-8864a74e0c74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:50.316236 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9dcb9-4df0-46fb-bec4-8864a74e0c74 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:50.353826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1057/2142] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:50 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:37:50.366349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d994192-e53f-459f-b1e4-7b9e8b2de39d service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:50.384111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d994192-e53f-459f-b1e4-7b9e8b2de39d service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:50.444985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:50.479948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712c883-1df9-4c0e-a67e-b98f743a769b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:50.481254 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712c883-1df9-4c0e-a67e-b98f743a769b admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:50.482499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712c883-1df9-4c0e-a67e-b98f743a769b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:50.482499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712c883-1df9-4c0e-a67e-b98f743a769b admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:50.483079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712c883-1df9-4c0e-a67e-b98f743a769b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:50.485983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712c883-1df9-4c0e-a67e-b98f743a769b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:50.519022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1086/2143] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:50 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:37:50.524511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9dcb9-4df0-46fb-bec4-8864a74e0c74 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:50.536190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9dcb9-4df0-46fb-bec4-8864a74e0c74 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:50.542096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add7651-171a-4b60-8e16-3c7e7efd7cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:50.542782 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add7651-171a-4b60-8e16-3c7e7efd7cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:50.543060 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add7651-171a-4b60-8e16-3c7e7efd7cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:50.543335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add7651-171a-4b60-8e16-3c7e7efd7cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:50.544076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add7651-171a-4b60-8e16-3c7e7efd7cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:50.545752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add7651-171a-4b60-8e16-3c7e7efd7cb1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:50.558880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1058/2144] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:50.563821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712c883-1df9-4c0e-a67e-b98f743a769b admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:50.575386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712c883-1df9-4c0e-a67e-b98f743a769b admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:50.581213 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d762272-5aca-49d6-a4db-3db326b97f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:50.582141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d762272-5aca-49d6-a4db-3db326b97f50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:50.582427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d762272-5aca-49d6-a4db-3db326b97f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:50.582704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d762272-5aca-49d6-a4db-3db326b97f50 admin admin] PATH_INFO: `/v3/projects/79680915af0744cda237e89728263bca/users/5ec1c01b3ceb46859b7e264647335c86/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:50.607710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d762272-5aca-49d6-a4db-3db326b97f50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79680915af0744cda237e89728263bca, user_id=5ec1c01b3ceb46859b7e264647335c86, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:50.616842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d762272-5aca-49d6-a4db-3db326b97f50 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:50.641682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1087/2145] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:50 2022] PUT /identity/v3/projects/79680915af0744cda237e89728263bca/users/5ec1c01b3ceb46859b7e264647335c86/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:37:50.648253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add7651-171a-4b60-8e16-3c7e7efd7cb1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:50.666407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add7651-171a-4b60-8e16-3c7e7efd7cb1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:50.750578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:50.804803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511376c8-9c90-4d51-82a7-499509c1fd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:50.805594 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511376c8-9c90-4d51-82a7-499509c1fd3f admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:50.805917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511376c8-9c90-4d51-82a7-499509c1fd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:50.806172 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511376c8-9c90-4d51-82a7-499509c1fd3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:50.806919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511376c8-9c90-4d51-82a7-499509c1fd3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:50.808548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511376c8-9c90-4d51-82a7-499509c1fd3f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:50.820653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1059/2146] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:37:50.824924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d762272-5aca-49d6-a4db-3db326b97f50 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:50.839228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d762272-5aca-49d6-a4db-3db326b97f50 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:50.844650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e753d-b7b1-4edf-a8c3-eece6999c459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:50.845775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e753d-b7b1-4edf-a8c3-eece6999c459 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:50.846192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e753d-b7b1-4edf-a8c3-eece6999c459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:50.846492 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e753d-b7b1-4edf-a8c3-eece6999c459 admin admin] PATH_INFO: `/v3/projects/79680915af0744cda237e89728263bca/users/5ec1c01b3ceb46859b7e264647335c86/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:50.848657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e753d-b7b1-4edf-a8c3-eece6999c459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79680915af0744cda237e89728263bca, user_id=5ec1c01b3ceb46859b7e264647335c86, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:50.852010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e753d-b7b1-4edf-a8c3-eece6999c459 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:50.864960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1088/2147] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:50 2022] PUT /identity/v3/projects/79680915af0744cda237e89728263bca/users/5ec1c01b3ceb46859b7e264647335c86/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:50.875481 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723f132-4a53-401e-b4cb-610888ebd5ac None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:50.875692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723f132-4a53-401e-b4cb-610888ebd5ac None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:50.875974 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723f132-4a53-401e-b4cb-610888ebd5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:50.935100 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7723f132-4a53-401e-b4cb-610888ebd5ac None None] MFA Rules not processed for user `5ec1c01b3ceb46859b7e264647335c86`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:50.937154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7723f132-4a53-401e-b4cb-610888ebd5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:51.077797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1060/2148] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:51.091614 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e753d-b7b1-4edf-a8c3-eece6999c459 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:51.179287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e753d-b7b1-4edf-a8c3-eece6999c459 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:51.183923 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df0f97-25f0-4364-a75a-f4a05723e45b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:51.184583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df0f97-25f0-4364-a75a-f4a05723e45b service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:51.184893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df0f97-25f0-4364-a75a-f4a05723e45b service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:51.185383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df0f97-25f0-4364-a75a-f4a05723e45b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:51.196952 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df0f97-25f0-4364-a75a-f4a05723e45b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:51.198494 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df0f97-25f0-4364-a75a-f4a05723e45b service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:51.223184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1089/2149] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:51.837063 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723f132-4a53-401e-b4cb-610888ebd5ac None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:51.848566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723f132-4a53-401e-b4cb-610888ebd5ac None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:51.854573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327d674-ac7d-4d8b-b0f3-518bb02f5f68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:51.855499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327d674-ac7d-4d8b-b0f3-518bb02f5f68 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:51.856544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327d674-ac7d-4d8b-b0f3-518bb02f5f68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:51.856864 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327d674-ac7d-4d8b-b0f3-518bb02f5f68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:51.960211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327d674-ac7d-4d8b-b0f3-518bb02f5f68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:51.964651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327d674-ac7d-4d8b-b0f3-518bb02f5f68 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:51.978204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1061/2150] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:53.090383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df0f97-25f0-4364-a75a-f4a05723e45b service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:53.172066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df0f97-25f0-4364-a75a-f4a05723e45b service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:53.178322 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:53.179462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:53.179462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:53.179756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] PATH_INFO: `/v3/users/5b68ff92655141df92b3470ffc6a493b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:53.181202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b68ff92655141df92b3470ffc6a493b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:53.183708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:53.237440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:53.248596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b68ff92655141df92b3470ffc6a493b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:53.249462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b68ff92655141df92b3470ffc6a493b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:53.263951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b68ff92655141df92b3470ffc6a493b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:53.281723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b68ff92655141df92b3470ffc6a493b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:53.282582 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b68ff92655141df92b3470ffc6a493b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:53.305346 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1090/2151] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:53 2022] DELETE /identity/v3/users/5b68ff92655141df92b3470ffc6a493b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:53.306978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bfc83-085a-4198-864e-33632a56a778 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:53.307446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bfc83-085a-4198-864e-33632a56a778 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:53.307536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bfc83-085a-4198-864e-33632a56a778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:53.335499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-046bfc83-085a-4198-864e-33632a56a778 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:53.337967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-046bfc83-085a-4198-864e-33632a56a778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:53.439078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:53.455084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1062/2152] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:53.472654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:53.477753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:53.573629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edc86f-6b8a-4754-abb4-e56ed00e6259 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:53.578838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca733e6-2364-41bb-89e5-e9510767c134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:53.580083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca733e6-2364-41bb-89e5-e9510767c134 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:53.581142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca733e6-2364-41bb-89e5-e9510767c134 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:53.581865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca733e6-2364-41bb-89e5-e9510767c134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:53.604539 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca733e6-2364-41bb-89e5-e9510767c134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:53.607549 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca733e6-2364-41bb-89e5-e9510767c134 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:53.621870 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1091/2153] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:53.964817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bfc83-085a-4198-864e-33632a56a778 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:53.966532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-046bfc83-085a-4198-864e-33632a56a778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:53.985694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bfc83-085a-4198-864e-33632a56a778 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:53.994999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:53.996667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:53.996973 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:53.997695 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f admin admin] PATH_INFO: `/v3/projects/e1ea6b05b673485f8c00c4f175878099` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:53.999908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1ea6b05b673485f8c00c4f175878099)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:54.003709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:54.067935 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1ea6b05b673485f8c00c4f175878099', 'request_id': 'req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:54.074862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1ea6b05b673485f8c00c4f175878099', 'request_id': 'req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:54.085796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1ea6b05b673485f8c00c4f175878099', 'request_id': 'req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:54.107855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f admin admin] The token cache is being invalidate because project e1ea6b05b673485f8c00c4f175878099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:37:54.108157 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:54.109915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2b1c2fc6-d86c-498e-9eaa-895f07aeea8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:54.113491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1063/2154] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:37:53 2022] DELETE /identity/v3/projects/e1ea6b05b673485f8c00c4f175878099 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:54.119877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04731f29-7f5e-42ee-beb8-bafe64ac2c98 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:54.120444 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04731f29-7f5e-42ee-beb8-bafe64ac2c98 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:54.120444 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04731f29-7f5e-42ee-beb8-bafe64ac2c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:54.140282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-04731f29-7f5e-42ee-beb8-bafe64ac2c98 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:54.141555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-04731f29-7f5e-42ee-beb8-bafe64ac2c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:54.233280 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1092/2155] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:54.243820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a2e26-76cd-477e-be96-8bb5b99ef336 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:54.244103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a2e26-76cd-477e-be96-8bb5b99ef336 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:54.244382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a2e26-76cd-477e-be96-8bb5b99ef336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:54.264861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-334a2e26-76cd-477e-be96-8bb5b99ef336 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:54.266697 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-334a2e26-76cd-477e-be96-8bb5b99ef336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:54.276550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1064/2156] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:37:54.283602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04731f29-7f5e-42ee-beb8-bafe64ac2c98 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:54.294629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04731f29-7f5e-42ee-beb8-bafe64ac2c98 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:54.299624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8f311-4100-4e54-b6dd-a62a8182b503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:54.300308 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8f311-4100-4e54-b6dd-a62a8182b503 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:54.300593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8f311-4100-4e54-b6dd-a62a8182b503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:54.300869 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8f311-4100-4e54-b6dd-a62a8182b503 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:54.301641 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8f311-4100-4e54-b6dd-a62a8182b503 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:37:54.302140 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8f311-4100-4e54-b6dd-a62a8182b503 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:54.303822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8f311-4100-4e54-b6dd-a62a8182b503 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:54.316327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1093/2157] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:37:54.322317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a2e26-76cd-477e-be96-8bb5b99ef336 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:54.332356 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a2e26-76cd-477e-be96-8bb5b99ef336 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:54.338769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a16762-c55b-4010-94ef-302dc21cff56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:54.339441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a16762-c55b-4010-94ef-302dc21cff56 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:54.339731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a16762-c55b-4010-94ef-302dc21cff56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:54.340011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a16762-c55b-4010-94ef-302dc21cff56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:54.340710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a16762-c55b-4010-94ef-302dc21cff56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:54.342667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a16762-c55b-4010-94ef-302dc21cff56 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:54.373708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1065/2158] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:37:54 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:54.378269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8f311-4100-4e54-b6dd-a62a8182b503 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:54.389643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8f311-4100-4e54-b6dd-a62a8182b503 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:54.436919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:54.453954 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6905ac-aa73-478f-a95b-6f9c11945ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:54.454638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6905ac-aa73-478f-a95b-6f9c11945ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:54.454925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6905ac-aa73-478f-a95b-6f9c11945ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:54.455205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6905ac-aa73-478f-a95b-6f9c11945ee0 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:54.455932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6905ac-aa73-478f-a95b-6f9c11945ee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:54.457896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6905ac-aa73-478f-a95b-6f9c11945ee0 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:54.480517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1094/2159] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:37:54 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:37:54.485053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a16762-c55b-4010-94ef-302dc21cff56 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:54.497627 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a16762-c55b-4010-94ef-302dc21cff56 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:54.502728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ccaae-789f-4573-957c-8ad656695ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:54.503401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ccaae-789f-4573-957c-8ad656695ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:54.503688 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ccaae-789f-4573-957c-8ad656695ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:54.503967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ccaae-789f-4573-957c-8ad656695ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:54.504699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ccaae-789f-4573-957c-8ad656695ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:54.507086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ccaae-789f-4573-957c-8ad656695ad2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:54.519138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1066/2160] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:37:54.524729 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6905ac-aa73-478f-a95b-6f9c11945ee0 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:54.537000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6905ac-aa73-478f-a95b-6f9c11945ee0 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:54.541863 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c118c0-10c5-4232-9d81-7e0acafc31f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:54.542548 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c118c0-10c5-4232-9d81-7e0acafc31f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:54.542860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c118c0-10c5-4232-9d81-7e0acafc31f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:54.543143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c118c0-10c5-4232-9d81-7e0acafc31f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:54.543882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c118c0-10c5-4232-9d81-7e0acafc31f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:54.545543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c118c0-10c5-4232-9d81-7e0acafc31f9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:54.561495 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1095/2161] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:37:54.566147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ccaae-789f-4573-957c-8ad656695ad2 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:54.578028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ccaae-789f-4573-957c-8ad656695ad2 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:54.584133 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4371c1d-80e6-4581-977e-590f7fc0e2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:54.585075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4371c1d-80e6-4581-977e-590f7fc0e2e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:54.585390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4371c1d-80e6-4581-977e-590f7fc0e2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:54.585670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4371c1d-80e6-4581-977e-590f7fc0e2e6 admin admin] PATH_INFO: `/v3/projects/bd7b2c1285f3499291d62b31c8a8111d/users/3bc2af409e9c4f629b2d20140cd1e81a/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:54.603806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4371c1d-80e6-4581-977e-590f7fc0e2e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd7b2c1285f3499291d62b31c8a8111d, user_id=3bc2af409e9c4f629b2d20140cd1e81a, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:54.606788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4371c1d-80e6-4581-977e-590f7fc0e2e6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:54.623890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1067/2162] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:37:54 2022] PUT /identity/v3/projects/bd7b2c1285f3499291d62b31c8a8111d/users/3bc2af409e9c4f629b2d20140cd1e81a/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:54.638585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b63aa8-d922-4d48-aa8b-23b57e092927 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:54.638585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b63aa8-d922-4d48-aa8b-23b57e092927 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:54.640667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b63aa8-d922-4d48-aa8b-23b57e092927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:54.676000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e8b63aa8-d922-4d48-aa8b-23b57e092927 None None] MFA Rules not processed for user `3bc2af409e9c4f629b2d20140cd1e81a`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:54.677361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e8b63aa8-d922-4d48-aa8b-23b57e092927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:54.803588 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1096/2163] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:54.815405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4371c1d-80e6-4581-977e-590f7fc0e2e6 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:54.818108 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a4371c1d-80e6-4581-977e-590f7fc0e2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:54.903257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4371c1d-80e6-4581-977e-590f7fc0e2e6 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:54.911254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8353f-c62d-45a9-ba7b-4ba1f0b4f7c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:54.912663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8353f-c62d-45a9-ba7b-4ba1f0b4f7c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:54.913403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8353f-c62d-45a9-ba7b-4ba1f0b4f7c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:54.914052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8353f-c62d-45a9-ba7b-4ba1f0b4f7c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:54.930075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8353f-c62d-45a9-ba7b-4ba1f0b4f7c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:54.932739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8353f-c62d-45a9-ba7b-4ba1f0b4f7c3 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:54.951097 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1068/2164] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:56.341701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b63aa8-d922-4d48-aa8b-23b57e092927 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:56.353452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b63aa8-d922-4d48-aa8b-23b57e092927 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:56.357371 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f628188-7dc0-4b86-a1bd-75e1050b1413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:56.358155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f628188-7dc0-4b86-a1bd-75e1050b1413 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:56.358382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f628188-7dc0-4b86-a1bd-75e1050b1413 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:56.358661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f628188-7dc0-4b86-a1bd-75e1050b1413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:56.360500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0f628188-7dc0-4b86-a1bd-75e1050b1413 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:56.431665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f628188-7dc0-4b86-a1bd-75e1050b1413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:56.433036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f628188-7dc0-4b86-a1bd-75e1050b1413 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:56.445376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1097/2165] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:37:59.058414 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8353f-c62d-45a9-ba7b-4ba1f0b4f7c3 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:59.061050 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-32c8353f-c62d-45a9-ba7b-4ba1f0b4f7c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:59.107901 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:59.206357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8353f-c62d-45a9-ba7b-4ba1f0b4f7c3 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:59.219370 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:59.222410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:59.223189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:59.224307 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] PATH_INFO: `/v3/users/dffe21f69a7c4ed7a20a0f206d40e27f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:59.228199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dffe21f69a7c4ed7a20a0f206d40e27f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:59.231844 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:59.368111 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dffe21f69a7c4ed7a20a0f206d40e27f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:59.370790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dffe21f69a7c4ed7a20a0f206d40e27f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:59.385704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dffe21f69a7c4ed7a20a0f206d40e27f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:59.387307 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dffe21f69a7c4ed7a20a0f206d40e27f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:59.408622 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dffe21f69a7c4ed7a20a0f206d40e27f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:37:59.433026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1069/2166] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:37:59 2022] DELETE /identity/v3/users/dffe21f69a7c4ed7a20a0f206d40e27f => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:37:59.436756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346dea2-7f39-4a84-af8c-9dabbdc91773 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:59.437388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346dea2-7f39-4a84-af8c-9dabbdc91773 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:59.437388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346dea2-7f39-4a84-af8c-9dabbdc91773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:59.458409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-6346dea2-7f39-4a84-af8c-9dabbdc91773 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:37:59.460045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6346dea2-7f39-4a84-af8c-9dabbdc91773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:59.524978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:37:59.540408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1098/2167] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:37:59.554547 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:37:59.557562 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:37:59.702180 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc172fc0-2ba6-4298-b778-8e09e10f27d6 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:37:59.709194 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dd0bf-aabf-415d-bd78-a3acf155c46e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:37:59.714386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dd0bf-aabf-415d-bd78-a3acf155c46e service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:37:59.714386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dd0bf-aabf-415d-bd78-a3acf155c46e service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:37:59.714386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dd0bf-aabf-415d-bd78-a3acf155c46e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:37:59.732892 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dd0bf-aabf-415d-bd78-a3acf155c46e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:37:59.734619 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dd0bf-aabf-415d-bd78-a3acf155c46e service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:37:59.769324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1070/2168] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:37:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:00.658873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346dea2-7f39-4a84-af8c-9dabbdc91773 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:00.663793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6346dea2-7f39-4a84-af8c-9dabbdc91773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:00.687633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346dea2-7f39-4a84-af8c-9dabbdc91773 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:00.695088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:00.695892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:00.696223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:00.696498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] PATH_INFO: `/v3/projects/5388981bda864839935d89cee9939a6c` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:00.698942 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5388981bda864839935d89cee9939a6c)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:00.701640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:00.817168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5388981bda864839935d89cee9939a6c', 'request_id': 'req-7e5230a1-0462-4fc6-aab2-82049b851da9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:00.827202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5388981bda864839935d89cee9939a6c', 'request_id': 'req-7e5230a1-0462-4fc6-aab2-82049b851da9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:00.838337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5388981bda864839935d89cee9939a6c', 'request_id': 'req-7e5230a1-0462-4fc6-aab2-82049b851da9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:00.855105 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] The token cache is being invalidate because project 5388981bda864839935d89cee9939a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:00.855686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:00.856609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:00.858522 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1099/2169] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:00 2022] DELETE /identity/v3/projects/5388981bda864839935d89cee9939a6c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:00.862535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dd0bf-aabf-415d-bd78-a3acf155c46e service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:00.874329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dd0bf-aabf-415d-bd78-a3acf155c46e service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:00.924588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:00.957280 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:00.959879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:00.960429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:00.960758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] PATH_INFO: `/v3/users/9bd46764b8cf4a1aae58f8ba4b813920` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:00.962238 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bd46764b8cf4a1aae58f8ba4b813920)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:00.964459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:01.046824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bd46764b8cf4a1aae58f8ba4b813920'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:01.048235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bd46764b8cf4a1aae58f8ba4b813920'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:01.066741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bd46764b8cf4a1aae58f8ba4b813920'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:01.068697 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bd46764b8cf4a1aae58f8ba4b813920'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:01.087487 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bd46764b8cf4a1aae58f8ba4b813920'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:01.104747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1071/2170] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:00 2022] DELETE /identity/v3/users/9bd46764b8cf4a1aae58f8ba4b813920 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:01.679008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:01.681598 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:01.715824 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5230a1-0462-4fc6-aab2-82049b851da9 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:01.799786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:01.800995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:01.802424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:01.802922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] PATH_INFO: `/v3/projects/61a30a10fad14c618aa6a4233dabecf4` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:01.804744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a30a10fad14c618aa6a4233dabecf4)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:01.806452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:01.937417 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a30a10fad14c618aa6a4233dabecf4', 'request_id': 'req-54fe0560-8702-4a78-9998-1d6793e008a6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:01.949952 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a30a10fad14c618aa6a4233dabecf4', 'request_id': 'req-54fe0560-8702-4a78-9998-1d6793e008a6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:01.967057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a30a10fad14c618aa6a4233dabecf4', 'request_id': 'req-54fe0560-8702-4a78-9998-1d6793e008a6'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:01.995717 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] The token cache is being invalidate because project 61a30a10fad14c618aa6a4233dabecf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:01.996482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:01.997486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:01.999258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1100/2171] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:01 2022] DELETE /identity/v3/projects/61a30a10fad14c618aa6a4233dabecf4 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:02.007265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:02.009698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:02.033207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf9cba-2eba-4859-9a1a-813f11876987 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:02.148046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620af1fe-3315-4190-94fd-59976d225491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:02.148919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620af1fe-3315-4190-94fd-59976d225491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:02.149518 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620af1fe-3315-4190-94fd-59976d225491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:02.149805 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620af1fe-3315-4190-94fd-59976d225491 admin admin] PATH_INFO: `/v3/users/5ec1c01b3ceb46859b7e264647335c86` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:02.151218 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620af1fe-3315-4190-94fd-59976d225491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ec1c01b3ceb46859b7e264647335c86)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:02.153266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620af1fe-3315-4190-94fd-59976d225491 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:02.245675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-620af1fe-3315-4190-94fd-59976d225491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ec1c01b3ceb46859b7e264647335c86'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:02.246399 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-620af1fe-3315-4190-94fd-59976d225491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ec1c01b3ceb46859b7e264647335c86'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:02.260815 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-620af1fe-3315-4190-94fd-59976d225491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ec1c01b3ceb46859b7e264647335c86'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:02.261752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-620af1fe-3315-4190-94fd-59976d225491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ec1c01b3ceb46859b7e264647335c86'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:02.269447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:02.288193 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-620af1fe-3315-4190-94fd-59976d225491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ec1c01b3ceb46859b7e264647335c86'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:02.308802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1072/2172] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:02 2022] DELETE /identity/v3/users/5ec1c01b3ceb46859b7e264647335c86 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:02.749420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:02.751638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:02.774678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe0560-8702-4a78-9998-1d6793e008a6 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:02.783164 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:02.878895 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26fe1d-810f-47c0-8c76-b7c1d824c36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:02.879879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26fe1d-810f-47c0-8c76-b7c1d824c36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:02.880169 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26fe1d-810f-47c0-8c76-b7c1d824c36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:02.880448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26fe1d-810f-47c0-8c76-b7c1d824c36f admin admin] PATH_INFO: `/v3/projects/79680915af0744cda237e89728263bca` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:02.882401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26fe1d-810f-47c0-8c76-b7c1d824c36f admin admin] RBAC: Authorizing `identity:delete_project(project_id=79680915af0744cda237e89728263bca)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:02.884415 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26fe1d-810f-47c0-8c76-b7c1d824c36f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:02.971390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-df26fe1d-810f-47c0-8c76-b7c1d824c36f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79680915af0744cda237e89728263bca', 'request_id': 'req-df26fe1d-810f-47c0-8c76-b7c1d824c36f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:02.987353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-df26fe1d-810f-47c0-8c76-b7c1d824c36f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79680915af0744cda237e89728263bca', 'request_id': 'req-df26fe1d-810f-47c0-8c76-b7c1d824c36f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:03.007294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-df26fe1d-810f-47c0-8c76-b7c1d824c36f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79680915af0744cda237e89728263bca', 'request_id': 'req-df26fe1d-810f-47c0-8c76-b7c1d824c36f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:03.025111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-df26fe1d-810f-47c0-8c76-b7c1d824c36f admin admin] The token cache is being invalidate because project 79680915af0744cda237e89728263bca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:03.025397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-df26fe1d-810f-47c0-8c76-b7c1d824c36f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:03.026068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-df26fe1d-810f-47c0-8c76-b7c1d824c36f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:03.029122 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1101/2173] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:02 2022] DELETE /identity/v3/projects/79680915af0744cda237e89728263bca => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:03.034233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d5d98-3d81-4bf1-bc47-2bc1f77f9820 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:03.034772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d5d98-3d81-4bf1-bc47-2bc1f77f9820 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:03.034772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d5d98-3d81-4bf1-bc47-2bc1f77f9820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:03.068234 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-360d5d98-3d81-4bf1-bc47-2bc1f77f9820 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:03.069940 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-360d5d98-3d81-4bf1-bc47-2bc1f77f9820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:03.159963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1073/2174] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:03.170159 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f922fc6-b2a0-4424-8ac9-d9d9dd38c16d None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:03.170521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f922fc6-b2a0-4424-8ac9-d9d9dd38c16d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:03.170747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f922fc6-b2a0-4424-8ac9-d9d9dd38c16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:03.199518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-7f922fc6-b2a0-4424-8ac9-d9d9dd38c16d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:03.200755 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-7f922fc6-b2a0-4424-8ac9-d9d9dd38c16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:03.217255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1102/2175] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:03.222037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d5d98-3d81-4bf1-bc47-2bc1f77f9820 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:03.239889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d5d98-3d81-4bf1-bc47-2bc1f77f9820 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:03.248069 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93af9b6-02f6-4586-81a5-fdcda68fbba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:03.249237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93af9b6-02f6-4586-81a5-fdcda68fbba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:03.249722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93af9b6-02f6-4586-81a5-fdcda68fbba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:03.250218 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93af9b6-02f6-4586-81a5-fdcda68fbba0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:03.251574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93af9b6-02f6-4586-81a5-fdcda68fbba0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:03.252148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93af9b6-02f6-4586-81a5-fdcda68fbba0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:03.255901 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93af9b6-02f6-4586-81a5-fdcda68fbba0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:03.278595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1074/2176] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:38:03.281437 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f922fc6-b2a0-4424-8ac9-d9d9dd38c16d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:03.296045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f922fc6-b2a0-4424-8ac9-d9d9dd38c16d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:03.301645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842f26c-561a-46d5-b53d-470357079954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:03.303318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842f26c-561a-46d5-b53d-470357079954 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:03.303318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842f26c-561a-46d5-b53d-470357079954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:03.303584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842f26c-561a-46d5-b53d-470357079954 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:03.304290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842f26c-561a-46d5-b53d-470357079954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:03.307212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842f26c-561a-46d5-b53d-470357079954 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:03.342889 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1103/2177] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:03 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:03.349975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93af9b6-02f6-4586-81a5-fdcda68fbba0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:03.369767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93af9b6-02f6-4586-81a5-fdcda68fbba0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:03.460816 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dc49d-84c1-4146-be22-edad1b716b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:03.461417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dc49d-84c1-4146-be22-edad1b716b63 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:03.461699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dc49d-84c1-4146-be22-edad1b716b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:03.461977 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dc49d-84c1-4146-be22-edad1b716b63 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:03.462675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159dc49d-84c1-4146-be22-edad1b716b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:03.464646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159dc49d-84c1-4146-be22-edad1b716b63 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:03.513746 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1075/2178] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:03 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:03.516143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842f26c-561a-46d5-b53d-470357079954 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:03.535700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842f26c-561a-46d5-b53d-470357079954 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:03.544209 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2305b-f86f-4cb4-a76c-339a2c8c4164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:03.545408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2305b-f86f-4cb4-a76c-339a2c8c4164 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:03.545899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2305b-f86f-4cb4-a76c-339a2c8c4164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:03.546344 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2305b-f86f-4cb4-a76c-339a2c8c4164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:03.547682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2305b-f86f-4cb4-a76c-339a2c8c4164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:03.550725 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2305b-f86f-4cb4-a76c-339a2c8c4164 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:03.566050 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1104/2179] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:03.570317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dc49d-84c1-4146-be22-edad1b716b63 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:03.580904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dc49d-84c1-4146-be22-edad1b716b63 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:03.589221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b1980-7b6d-406c-a854-155ebe15d103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:03.589560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b1980-7b6d-406c-a854-155ebe15d103 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:03.589868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b1980-7b6d-406c-a854-155ebe15d103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:03.590113 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b1980-7b6d-406c-a854-155ebe15d103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:03.590921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b1980-7b6d-406c-a854-155ebe15d103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:03.592951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b1980-7b6d-406c-a854-155ebe15d103 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:03.608371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1076/2180] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:38:03.613056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2305b-f86f-4cb4-a76c-339a2c8c4164 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:03.624878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2305b-f86f-4cb4-a76c-339a2c8c4164 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:03.629862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495f737-f423-41d6-850a-cdf9f11a5cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:03.631423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495f737-f423-41d6-850a-cdf9f11a5cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:03.631771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495f737-f423-41d6-850a-cdf9f11a5cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:03.632048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495f737-f423-41d6-850a-cdf9f11a5cc4 admin admin] PATH_INFO: `/v3/projects/06f76285ca404a458ddba5808b56a7c0/users/fdaf9c411f5b445397078958a760509a/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:03.651255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495f737-f423-41d6-850a-cdf9f11a5cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f76285ca404a458ddba5808b56a7c0, user_id=fdaf9c411f5b445397078958a760509a, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:03.653984 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495f737-f423-41d6-850a-cdf9f11a5cc4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:03.670436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1105/2181] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:03 2022] PUT /identity/v3/projects/06f76285ca404a458ddba5808b56a7c0/users/fdaf9c411f5b445397078958a760509a/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:03.682415 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9802d8-06e1-45eb-91a9-b95544e0eb46 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:03.682842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9802d8-06e1-45eb-91a9-b95544e0eb46 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:03.683250 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9802d8-06e1-45eb-91a9-b95544e0eb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:03.718328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-fb9802d8-06e1-45eb-91a9-b95544e0eb46 None None] MFA Rules not processed for user `fdaf9c411f5b445397078958a760509a`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:03.719721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fb9802d8-06e1-45eb-91a9-b95544e0eb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:03.745400 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:03.806199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1077/2182] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:03.811222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495f737-f423-41d6-850a-cdf9f11a5cc4 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:03.853150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:03.887038 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495f737-f423-41d6-850a-cdf9f11a5cc4 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:03.897150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8b412-3f24-4342-a96f-7a6ddd43eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:03.897150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8b412-3f24-4342-a96f-7a6ddd43eb99 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:03.897150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8b412-3f24-4342-a96f-7a6ddd43eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:03.897150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8b412-3f24-4342-a96f-7a6ddd43eb99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:03.897820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8b412-3f24-4342-a96f-7a6ddd43eb99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:03.898662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8b412-3f24-4342-a96f-7a6ddd43eb99 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:03.935207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1106/2183] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:03 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:03.939865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9802d8-06e1-45eb-91a9-b95544e0eb46 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:03.961041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9802d8-06e1-45eb-91a9-b95544e0eb46 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.038297 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9622cd-dae5-41cb-bcab-b17087a60373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.039476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9622cd-dae5-41cb-bcab-b17087a60373 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.039972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9622cd-dae5-41cb-bcab-b17087a60373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.040490 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9622cd-dae5-41cb-bcab-b17087a60373 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.042011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9622cd-dae5-41cb-bcab-b17087a60373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.045929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9622cd-dae5-41cb-bcab-b17087a60373 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.075381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1078/2184] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:03 2022] POST /identity/v3/users => generated 297 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:04.080274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8b412-3f24-4342-a96f-7a6ddd43eb99 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.101321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8b412-3f24-4342-a96f-7a6ddd43eb99 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.108244 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557dea9-d69b-43f9-b483-c99619d5bd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.108956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557dea9-d69b-43f9-b483-c99619d5bd29 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.109292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557dea9-d69b-43f9-b483-c99619d5bd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.109578 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557dea9-d69b-43f9-b483-c99619d5bd29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.110326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557dea9-d69b-43f9-b483-c99619d5bd29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.112006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557dea9-d69b-43f9-b483-c99619d5bd29 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.129328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1107/2185] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:04.134493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9622cd-dae5-41cb-bcab-b17087a60373 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.148642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9622cd-dae5-41cb-bcab-b17087a60373 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.158533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25620b08-d92e-4111-b33a-c441c4410173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.160108 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25620b08-d92e-4111-b33a-c441c4410173 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.160579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25620b08-d92e-4111-b33a-c441c4410173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.161705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25620b08-d92e-4111-b33a-c441c4410173 admin admin] PATH_INFO: `/v3/projects/d94173ccaa744155ac102bb662fef79f/users/d5fcb07e5a4a48a3bc2a25b651612813/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.189072 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25620b08-d92e-4111-b33a-c441c4410173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d94173ccaa744155ac102bb662fef79f, user_id=d5fcb07e5a4a48a3bc2a25b651612813, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.191451 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25620b08-d92e-4111-b33a-c441c4410173 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.218201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1079/2186] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:04 2022] PUT /identity/v3/projects/d94173ccaa744155ac102bb662fef79f/users/d5fcb07e5a4a48a3bc2a25b651612813/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:04.224261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557dea9-d69b-43f9-b483-c99619d5bd29 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.243265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557dea9-d69b-43f9-b483-c99619d5bd29 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.330228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec13b3c-a518-4aaf-8909-96ebff457478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.331039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec13b3c-a518-4aaf-8909-96ebff457478 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.331397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec13b3c-a518-4aaf-8909-96ebff457478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.331737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec13b3c-a518-4aaf-8909-96ebff457478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.332555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec13b3c-a518-4aaf-8909-96ebff457478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.339261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec13b3c-a518-4aaf-8909-96ebff457478 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.351708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1108/2187] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:04.357392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25620b08-d92e-4111-b33a-c441c4410173 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.382060 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25620b08-d92e-4111-b33a-c441c4410173 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.392972 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50245a53-1637-457a-8b03-14fdf1f2970e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.394861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50245a53-1637-457a-8b03-14fdf1f2970e admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.396367 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50245a53-1637-457a-8b03-14fdf1f2970e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.396889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50245a53-1637-457a-8b03-14fdf1f2970e admin admin] PATH_INFO: `/v3/projects/d94173ccaa744155ac102bb662fef79f/users/d5fcb07e5a4a48a3bc2a25b651612813/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.402113 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50245a53-1637-457a-8b03-14fdf1f2970e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d94173ccaa744155ac102bb662fef79f, user_id=d5fcb07e5a4a48a3bc2a25b651612813, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.408548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50245a53-1637-457a-8b03-14fdf1f2970e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.433160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1080/2188] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:04 2022] PUT /identity/v3/projects/d94173ccaa744155ac102bb662fef79f/users/d5fcb07e5a4a48a3bc2a25b651612813/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:04.452767 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946da9e-c955-4910-8c8b-b9bf83450915 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.453121 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946da9e-c955-4910-8c8b-b9bf83450915 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.453447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946da9e-c955-4910-8c8b-b9bf83450915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.483962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-4946da9e-c955-4910-8c8b-b9bf83450915 None None] MFA Rules not processed for user `d5fcb07e5a4a48a3bc2a25b651612813`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:04.485261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4946da9e-c955-4910-8c8b-b9bf83450915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.578173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1109/2189] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:04.588850 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50245a53-1637-457a-8b03-14fdf1f2970e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.591384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-50245a53-1637-457a-8b03-14fdf1f2970e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.676076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50245a53-1637-457a-8b03-14fdf1f2970e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.680728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540515d-39fc-4be7-b02d-8578105eb24d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.681418 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540515d-39fc-4be7-b02d-8578105eb24d service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.681704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540515d-39fc-4be7-b02d-8578105eb24d service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.682022 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540515d-39fc-4be7-b02d-8578105eb24d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.691903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540515d-39fc-4be7-b02d-8578105eb24d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.693646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540515d-39fc-4be7-b02d-8578105eb24d service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.704661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1081/2190] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:04.807057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946da9e-c955-4910-8c8b-b9bf83450915 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.824305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946da9e-c955-4910-8c8b-b9bf83450915 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.828151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed85811-d67e-43a8-bd43-4d1e8e662b8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.828810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed85811-d67e-43a8-bd43-4d1e8e662b8e service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.829622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed85811-d67e-43a8-bd43-4d1e8e662b8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.829901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed85811-d67e-43a8-bd43-4d1e8e662b8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.867984 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:04.929612 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed85811-d67e-43a8-bd43-4d1e8e662b8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.932417 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed85811-d67e-43a8-bd43-4d1e8e662b8e service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.954045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1110/2191] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:05.284895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540515d-39fc-4be7-b02d-8578105eb24d service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.307765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:05.356153 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540515d-39fc-4be7-b02d-8578105eb24d service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.361012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.361927 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.362204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.362477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] PATH_INFO: `/v3/users/fdaf9c411f5b445397078958a760509a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.363725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdaf9c411f5b445397078958a760509a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.366380 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.426300 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdaf9c411f5b445397078958a760509a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:05.427316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdaf9c411f5b445397078958a760509a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:05.440100 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdaf9c411f5b445397078958a760509a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:05.441204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdaf9c411f5b445397078958a760509a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:05.461810 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdaf9c411f5b445397078958a760509a'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:05.477666 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1082/2192] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:05 2022] DELETE /identity/v3/users/fdaf9c411f5b445397078958a760509a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:05.482264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fe0b8-600c-4478-b043-3b33ca55002d None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.482684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fe0b8-600c-4478-b043-3b33ca55002d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.483034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fe0b8-600c-4478-b043-3b33ca55002d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.502951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-020fe0b8-600c-4478-b043-3b33ca55002d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:05.506020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-020fe0b8-600c-4478-b043-3b33ca55002d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:05.600553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1111/2193] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:05.612367 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.613744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:05.687085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109f017-f2a2-402a-8051-1982cbc42a91 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.691627 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831c0c9-7743-46d5-911b-70e44beb6a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.693207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831c0c9-7743-46d5-911b-70e44beb6a94 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.693506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831c0c9-7743-46d5-911b-70e44beb6a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.693805 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831c0c9-7743-46d5-911b-70e44beb6a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.703383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831c0c9-7743-46d5-911b-70e44beb6a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.704810 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831c0c9-7743-46d5-911b-70e44beb6a94 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.716633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1083/2194] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:06.040283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fe0b8-600c-4478-b043-3b33ca55002d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.042921 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-020fe0b8-600c-4478-b043-3b33ca55002d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:06.091805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fe0b8-600c-4478-b043-3b33ca55002d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.106792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.111830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.112326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.115028 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] PATH_INFO: `/v3/projects/06f76285ca404a458ddba5808b56a7c0` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.117977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06f76285ca404a458ddba5808b56a7c0)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.119862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.206835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06f76285ca404a458ddba5808b56a7c0', 'request_id': 'req-e2120e3d-3312-4f5e-a33a-16842ead44b1'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.218497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06f76285ca404a458ddba5808b56a7c0', 'request_id': 'req-e2120e3d-3312-4f5e-a33a-16842ead44b1'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.231267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06f76285ca404a458ddba5808b56a7c0', 'request_id': 'req-e2120e3d-3312-4f5e-a33a-16842ead44b1'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.244008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] The token cache is being invalidate because project 06f76285ca404a458ddba5808b56a7c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:06.244305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.245006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.246764 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1112/2195] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:06 2022] DELETE /identity/v3/projects/06f76285ca404a458ddba5808b56a7c0 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:06.250715 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831c0c9-7743-46d5-911b-70e44beb6a94 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.261746 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831c0c9-7743-46d5-911b-70e44beb6a94 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.313791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:06.339677 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.340900 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.341322 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.341699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] PATH_INFO: `/v3/users/d5fcb07e5a4a48a3bc2a25b651612813` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.343318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5fcb07e5a4a48a3bc2a25b651612813)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.346107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.407534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5fcb07e5a4a48a3bc2a25b651612813'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.408243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5fcb07e5a4a48a3bc2a25b651612813'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.416511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5fcb07e5a4a48a3bc2a25b651612813'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.417290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5fcb07e5a4a48a3bc2a25b651612813'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.430926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5fcb07e5a4a48a3bc2a25b651612813'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.446352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1084/2196] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:06 2022] DELETE /identity/v3/users/d5fcb07e5a4a48a3bc2a25b651612813 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:06.594011 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.595707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:06.610097 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2120e3d-3312-4f5e-a33a-16842ead44b1 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.662498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:06.675411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae5500-9ac0-4e12-9925-71cfa9da08a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.677907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae5500-9ac0-4e12-9925-71cfa9da08a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.678242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae5500-9ac0-4e12-9925-71cfa9da08a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.678568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae5500-9ac0-4e12-9925-71cfa9da08a0 admin admin] PATH_INFO: `/v3/users/3bc2af409e9c4f629b2d20140cd1e81a` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.680002 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae5500-9ac0-4e12-9925-71cfa9da08a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bc2af409e9c4f629b2d20140cd1e81a)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.682184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae5500-9ac0-4e12-9925-71cfa9da08a0 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.750767 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-13ae5500-9ac0-4e12-9925-71cfa9da08a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bc2af409e9c4f629b2d20140cd1e81a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.751529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-13ae5500-9ac0-4e12-9925-71cfa9da08a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bc2af409e9c4f629b2d20140cd1e81a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.760872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-13ae5500-9ac0-4e12-9925-71cfa9da08a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bc2af409e9c4f629b2d20140cd1e81a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.767320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.770889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:06.775318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-13ae5500-9ac0-4e12-9925-71cfa9da08a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bc2af409e9c4f629b2d20140cd1e81a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.776735 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-13ae5500-9ac0-4e12-9925-71cfa9da08a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bc2af409e9c4f629b2d20140cd1e81a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.795841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1113/2197] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:06 2022] DELETE /identity/v3/users/3bc2af409e9c4f629b2d20140cd1e81a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:06.801002 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6e5b1-b1c7-47d8-b999-64812d72cec5 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.801462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6e5b1-b1c7-47d8-b999-64812d72cec5 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.802461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6e5b1-b1c7-47d8-b999-64812d72cec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.822680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-edf6e5b1-b1c7-47d8-b999-64812d72cec5 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:06.824582 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-edf6e5b1-b1c7-47d8-b999-64812d72cec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:06.839108 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e8e4a-99c8-475b-be5a-33a980136ead admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.901321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1114/2198] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:06.912525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6e5b1-b1c7-47d8-b999-64812d72cec5 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.914022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-edf6e5b1-b1c7-47d8-b999-64812d72cec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:06.917515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.918409 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.918721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.919008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5 admin admin] PATH_INFO: `/v3/projects/d94173ccaa744155ac102bb662fef79f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.920265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d94173ccaa744155ac102bb662fef79f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.922220 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.974469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd94173ccaa744155ac102bb662fef79f', 'request_id': 'req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.976081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6e5b1-b1c7-47d8-b999-64812d72cec5 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.979858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd94173ccaa744155ac102bb662fef79f', 'request_id': 'req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:06.982397 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:06.990313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd94173ccaa744155ac102bb662fef79f', 'request_id': 'req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:07.006552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:07.006749 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5 admin admin] The token cache is being invalidate because project d94173ccaa744155ac102bb662fef79f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:07.007041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:07.007732 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-be15bbdd-3e10-4ac2-ad37-6e55074f3be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:07.009480 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1085/2199] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:06 2022] DELETE /identity/v3/projects/d94173ccaa744155ac102bb662fef79f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:07.014409 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df852191-3b79-42ef-abf3-782513a9231e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.014698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df852191-3b79-42ef-abf3-782513a9231e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.015002 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df852191-3b79-42ef-abf3-782513a9231e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.038120 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-df852191-3b79-42ef-abf3-782513a9231e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:07.038120 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-df852191-3b79-42ef-abf3-782513a9231e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.038651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321b93d-08b2-427c-9bd4-e84a4356fc9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.038651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321b93d-08b2-427c-9bd4-e84a4356fc9f service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.039086 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321b93d-08b2-427c-9bd4-e84a4356fc9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.039376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321b93d-08b2-427c-9bd4-e84a4356fc9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.041298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4321b93d-08b2-427c-9bd4-e84a4356fc9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.107817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1086/2200] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:07.116627 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa2d24-a31a-478a-9551-6b9ac85e15b0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.116854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321b93d-08b2-427c-9bd4-e84a4356fc9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.117006 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa2d24-a31a-478a-9551-6b9ac85e15b0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.117276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa2d24-a31a-478a-9551-6b9ac85e15b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.118295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321b93d-08b2-427c-9bd4-e84a4356fc9f service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.130342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1115/2201] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:38:07.135924 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-dcfa2d24-a31a-478a-9551-6b9ac85e15b0 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:07.137224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dcfa2d24-a31a-478a-9551-6b9ac85e15b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.147058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1087/2202] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:07.150831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321b93d-08b2-427c-9bd4-e84a4356fc9f service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.160753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321b93d-08b2-427c-9bd4-e84a4356fc9f service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.171921 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b57c2-f374-48f1-a29a-47da31ef2255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.174664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b57c2-f374-48f1-a29a-47da31ef2255 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.174985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b57c2-f374-48f1-a29a-47da31ef2255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.175332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b57c2-f374-48f1-a29a-47da31ef2255 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.176477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b57c2-f374-48f1-a29a-47da31ef2255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:07.176827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b57c2-f374-48f1-a29a-47da31ef2255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.179056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b57c2-f374-48f1-a29a-47da31ef2255 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.197027 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1116/2203] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:07.198146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa2d24-a31a-478a-9551-6b9ac85e15b0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.209119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa2d24-a31a-478a-9551-6b9ac85e15b0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.214344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959ba01-e65f-44cd-b721-4729fd61398f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.215011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959ba01-e65f-44cd-b721-4729fd61398f admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.215300 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959ba01-e65f-44cd-b721-4729fd61398f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.215576 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959ba01-e65f-44cd-b721-4729fd61398f admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.216271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959ba01-e65f-44cd-b721-4729fd61398f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.218235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959ba01-e65f-44cd-b721-4729fd61398f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.250157 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1088/2204] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:07 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:07.254594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b57c2-f374-48f1-a29a-47da31ef2255 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.271214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b57c2-f374-48f1-a29a-47da31ef2255 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.373128 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4df511-5e4c-4e1d-a624-3ebe0ad01b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.373790 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4df511-5e4c-4e1d-a624-3ebe0ad01b47 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.374055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4df511-5e4c-4e1d-a624-3ebe0ad01b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.374341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4df511-5e4c-4e1d-a624-3ebe0ad01b47 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.375248 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4df511-5e4c-4e1d-a624-3ebe0ad01b47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.379864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4df511-5e4c-4e1d-a624-3ebe0ad01b47 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.405148 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1117/2205] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:07 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:38:07.409218 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959ba01-e65f-44cd-b721-4729fd61398f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.422796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959ba01-e65f-44cd-b721-4729fd61398f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.431547 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c74ca1-23f8-4f9c-8a77-2614193df25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.432691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c74ca1-23f8-4f9c-8a77-2614193df25a admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.433390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c74ca1-23f8-4f9c-8a77-2614193df25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.433849 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c74ca1-23f8-4f9c-8a77-2614193df25a admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.434983 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c74ca1-23f8-4f9c-8a77-2614193df25a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.436838 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c74ca1-23f8-4f9c-8a77-2614193df25a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.452146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1089/2206] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:38:07.453466 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4df511-5e4c-4e1d-a624-3ebe0ad01b47 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.464736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4df511-5e4c-4e1d-a624-3ebe0ad01b47 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.475921 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bf30f-3718-4849-acc4-4fde48bcd893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.476549 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bf30f-3718-4849-acc4-4fde48bcd893 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.476835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bf30f-3718-4849-acc4-4fde48bcd893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.477131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bf30f-3718-4849-acc4-4fde48bcd893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.477898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51bf30f-3718-4849-acc4-4fde48bcd893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.479695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51bf30f-3718-4849-acc4-4fde48bcd893 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.496595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1118/2207] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:07.498364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c74ca1-23f8-4f9c-8a77-2614193df25a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.510120 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c74ca1-23f8-4f9c-8a77-2614193df25a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.515778 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccb9dc-c351-4d43-8c42-7c174de2f427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.517570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccb9dc-c351-4d43-8c42-7c174de2f427 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.517873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccb9dc-c351-4d43-8c42-7c174de2f427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.518200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccb9dc-c351-4d43-8c42-7c174de2f427 admin admin] PATH_INFO: `/v3/projects/b6d278430459441db81df74d79a4aa20/users/6602673b1a9c49af9daeacc80df5be44/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.539160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccb9dc-c351-4d43-8c42-7c174de2f427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d278430459441db81df74d79a4aa20, user_id=6602673b1a9c49af9daeacc80df5be44, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.544236 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccb9dc-c351-4d43-8c42-7c174de2f427 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.560401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bf30f-3718-4849-acc4-4fde48bcd893 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.565132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d51bf30f-3718-4849-acc4-4fde48bcd893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.577126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1090/2208] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:07 2022] PUT /identity/v3/projects/b6d278430459441db81df74d79a4aa20/users/6602673b1a9c49af9daeacc80df5be44/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:07.588505 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2f1cd-e85c-4f8b-97b6-bbcf8589d4d6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.588505 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2f1cd-e85c-4f8b-97b6-bbcf8589d4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.588809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2f1cd-e85c-4f8b-97b6-bbcf8589d4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.593564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bf30f-3718-4849-acc4-4fde48bcd893 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.619633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-54f2f1cd-e85c-4f8b-97b6-bbcf8589d4d6 None None] MFA Rules not processed for user `6602673b1a9c49af9daeacc80df5be44`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:07.620865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-54f2f1cd-e85c-4f8b-97b6-bbcf8589d4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.647870 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:07.667349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.668252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.668542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.668822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443 admin admin] PATH_INFO: `/v3/projects/bd7b2c1285f3499291d62b31c8a8111d` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.670097 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd7b2c1285f3499291d62b31c8a8111d)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.672418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.705589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1091/2209] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:07.722883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2f1cd-e85c-4f8b-97b6-bbcf8589d4d6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.725170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-54f2f1cd-e85c-4f8b-97b6-bbcf8589d4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.729644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd7b2c1285f3499291d62b31c8a8111d', 'request_id': 'req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:07.738758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd7b2c1285f3499291d62b31c8a8111d', 'request_id': 'req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:07.748021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:07.748852 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd7b2c1285f3499291d62b31c8a8111d', 'request_id': 'req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:07.764272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443 admin admin] The token cache is being invalidate because project bd7b2c1285f3499291d62b31c8a8111d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:07.764619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:07.765524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3a9b3ef3-fed9-4018-ab87-7eb11fe45443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:07.768122 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1119/2210] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:07 2022] DELETE /identity/v3/projects/bd7b2c1285f3499291d62b31c8a8111d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:07.782927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a950d3d-5601-4f0a-b675-11160ad4255a None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.783289 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a950d3d-5601-4f0a-b675-11160ad4255a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.783691 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a950d3d-5601-4f0a-b675-11160ad4255a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.808665 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2f1cd-e85c-4f8b-97b6-bbcf8589d4d6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.811402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-8a950d3d-5601-4f0a-b675-11160ad4255a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:07.813411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8a950d3d-5601-4f0a-b675-11160ad4255a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.909881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80756e-1a2f-45a2-a5ff-9113a723f3b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.910903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80756e-1a2f-45a2-a5ff-9113a723f3b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.911404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80756e-1a2f-45a2-a5ff-9113a723f3b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.911628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80756e-1a2f-45a2-a5ff-9113a723f3b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.914041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3c80756e-1a2f-45a2-a5ff-9113a723f3b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.954724 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1120/2211] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:07.966950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b217b-6cbb-4405-8237-94090c852e5d None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.967962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b217b-6cbb-4405-8237-94090c852e5d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.968548 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b217b-6cbb-4405-8237-94090c852e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.000894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-390b217b-6cbb-4405-8237-94090c852e5d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:08.002499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-390b217b-6cbb-4405-8237-94090c852e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:08.014274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80756e-1a2f-45a2-a5ff-9113a723f3b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.016288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80756e-1a2f-45a2-a5ff-9113a723f3b3 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.019219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1121/2212] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:08.023124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b217b-6cbb-4405-8237-94090c852e5d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.029345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1092/2213] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:38:08.036072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b217b-6cbb-4405-8237-94090c852e5d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.041303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b87704-4f33-4d0b-bf92-0d7d248360a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.041995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b87704-4f33-4d0b-bf92-0d7d248360a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.042258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b87704-4f33-4d0b-bf92-0d7d248360a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.042535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b87704-4f33-4d0b-bf92-0d7d248360a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.044286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b87704-4f33-4d0b-bf92-0d7d248360a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:08.044286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b87704-4f33-4d0b-bf92-0d7d248360a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.045262 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b87704-4f33-4d0b-bf92-0d7d248360a3 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.059154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1122/2214] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:08.063825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80756e-1a2f-45a2-a5ff-9113a723f3b3 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.074573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80756e-1a2f-45a2-a5ff-9113a723f3b3 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.080568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb131e-a15c-45cc-baf7-66fb9f7db38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.081282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb131e-a15c-45cc-baf7-66fb9f7db38a admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.081568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb131e-a15c-45cc-baf7-66fb9f7db38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.081846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb131e-a15c-45cc-baf7-66fb9f7db38a admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.083417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb131e-a15c-45cc-baf7-66fb9f7db38a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.085566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb131e-a15c-45cc-baf7-66fb9f7db38a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.118841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1093/2215] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:08 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:38:08.122834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b87704-4f33-4d0b-bf92-0d7d248360a3 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.132829 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b87704-4f33-4d0b-bf92-0d7d248360a3 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.191715 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:08.206607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689bace-0cd1-4688-bd8a-fe9bb372d608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.208633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689bace-0cd1-4688-bd8a-fe9bb372d608 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.209159 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689bace-0cd1-4688-bd8a-fe9bb372d608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.209632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689bace-0cd1-4688-bd8a-fe9bb372d608 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.210860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689bace-0cd1-4688-bd8a-fe9bb372d608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.213491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689bace-0cd1-4688-bd8a-fe9bb372d608 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.242004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1123/2216] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:08 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:08.248889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb131e-a15c-45cc-baf7-66fb9f7db38a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.261461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb131e-a15c-45cc-baf7-66fb9f7db38a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.267575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f246825-2f9b-477e-9ff9-f8ff3aebadfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.268739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f246825-2f9b-477e-9ff9-f8ff3aebadfb admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.269015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f246825-2f9b-477e-9ff9-f8ff3aebadfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.269495 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f246825-2f9b-477e-9ff9-f8ff3aebadfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.270117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f246825-2f9b-477e-9ff9-f8ff3aebadfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.272785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f246825-2f9b-477e-9ff9-f8ff3aebadfb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.286639 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1094/2217] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:08.290332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689bace-0cd1-4688-bd8a-fe9bb372d608 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.307390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689bace-0cd1-4688-bd8a-fe9bb372d608 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.314023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34908d90-c3d8-4caa-a03f-56d1c7b0675c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.315079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34908d90-c3d8-4caa-a03f-56d1c7b0675c admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.315469 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34908d90-c3d8-4caa-a03f-56d1c7b0675c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.315888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34908d90-c3d8-4caa-a03f-56d1c7b0675c admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.316747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34908d90-c3d8-4caa-a03f-56d1c7b0675c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.319086 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34908d90-c3d8-4caa-a03f-56d1c7b0675c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.331462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1124/2218] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:08.337051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f246825-2f9b-477e-9ff9-f8ff3aebadfb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.361135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f246825-2f9b-477e-9ff9-f8ff3aebadfb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.372802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5c25b-3407-41a7-a6cc-198b876097fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.373740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5c25b-3407-41a7-a6cc-198b876097fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.380424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5c25b-3407-41a7-a6cc-198b876097fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.381616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5c25b-3407-41a7-a6cc-198b876097fe admin admin] PATH_INFO: `/v3/projects/e3b94ee628c242599b57c104cb97bc86/users/5caffe8082c246c7b00f1eaa98154c46/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.417862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5c25b-3407-41a7-a6cc-198b876097fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b94ee628c242599b57c104cb97bc86, user_id=5caffe8082c246c7b00f1eaa98154c46, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.420740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5c25b-3407-41a7-a6cc-198b876097fe admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.445772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1095/2219] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:08 2022] PUT /identity/v3/projects/e3b94ee628c242599b57c104cb97bc86/users/5caffe8082c246c7b00f1eaa98154c46/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:08.449223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b24416-7898-4258-b94b-3d566f5ee23b None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.449454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b24416-7898-4258-b94b-3d566f5ee23b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.450135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b24416-7898-4258-b94b-3d566f5ee23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.483809 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-21b24416-7898-4258-b94b-3d566f5ee23b None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:08.485259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-21b24416-7898-4258-b94b-3d566f5ee23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:08.575977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1125/2220] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:08.592807 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5c25b-3407-41a7-a6cc-198b876097fe admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.594558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6fd5c25b-3407-41a7-a6cc-198b876097fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:08.628114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:08.654904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5c25b-3407-41a7-a6cc-198b876097fe admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.658672 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3e1f1-18df-4b70-8d39-66b92c2d3982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.659323 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3e1f1-18df-4b70-8d39-66b92c2d3982 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.659605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3e1f1-18df-4b70-8d39-66b92c2d3982 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.659884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3e1f1-18df-4b70-8d39-66b92c2d3982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.669706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3e1f1-18df-4b70-8d39-66b92c2d3982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.671128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3e1f1-18df-4b70-8d39-66b92c2d3982 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.681513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1096/2221] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:09.573285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d972d0-f55b-4daa-9e4a-a8ed5a1d2cdf None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:09.574259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d972d0-f55b-4daa-9e4a-a8ed5a1d2cdf None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:09.575097 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d972d0-f55b-4daa-9e4a-a8ed5a1d2cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:09.636846 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d9d972d0-f55b-4daa-9e4a-a8ed5a1d2cdf None None] MFA Rules not processed for user `5caffe8082c246c7b00f1eaa98154c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:09.639086 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d9d972d0-f55b-4daa-9e4a-a8ed5a1d2cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:09.761980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1126/2222] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:09.780594 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3e1f1-18df-4b70-8d39-66b92c2d3982 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:09.782261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-11e3e1f1-18df-4b70-8d39-66b92c2d3982 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:09.938921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3e1f1-18df-4b70-8d39-66b92c2d3982 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:09.947523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cb2ac-6ad0-4a7f-b3db-d9f9482d9cca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:09.948381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cb2ac-6ad0-4a7f-b3db-d9f9482d9cca service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:09.948666 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cb2ac-6ad0-4a7f-b3db-d9f9482d9cca service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:09.948941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cb2ac-6ad0-4a7f-b3db-d9f9482d9cca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:09.962415 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447cb2ac-6ad0-4a7f-b3db-d9f9482d9cca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:09.963883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447cb2ac-6ad0-4a7f-b3db-d9f9482d9cca service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:09.988081 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1097/2223] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:38:11.059707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c6598-56e0-4036-b90e-adac3232ce54 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:11.060217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c6598-56e0-4036-b90e-adac3232ce54 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:11.060331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c6598-56e0-4036-b90e-adac3232ce54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:11.078523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-366c6598-56e0-4036-b90e-adac3232ce54 None None] MFA Rules not processed for user `3e6fe6fa3fc14cca97064b49403eb68b`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:11.079762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-366c6598-56e0-4036-b90e-adac3232ce54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:11.136394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1127/2224] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:13.521491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c78cc4-e076-4140-8e8c-1ed176027eed None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:13.523255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c78cc4-e076-4140-8e8c-1ed176027eed None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:13.523764 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c78cc4-e076-4140-8e8c-1ed176027eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:13.553828 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f5c78cc4-e076-4140-8e8c-1ed176027eed None None] MFA Rules not processed for user `5caffe8082c246c7b00f1eaa98154c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:13.555277 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f5c78cc4-e076-4140-8e8c-1ed176027eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:13.568244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1098/2225] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:17.170001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f7724-0e7e-4e8d-9ac6-5fc3f5d625d4 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:17.170586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f7724-0e7e-4e8d-9ac6-5fc3f5d625d4 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:17.170668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f7724-0e7e-4e8d-9ac6-5fc3f5d625d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:17.182631 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:17.206004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-8f7f7724-0e7e-4e8d-9ac6-5fc3f5d625d4 None None] MFA Rules not processed for user `26d6eb06b23b4db88261ae58d650736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:17.207239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8f7f7724-0e7e-4e8d-9ac6-5fc3f5d625d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:17.283839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1128/2226] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jun 7 08:38:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:17.341986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c78cc4-e076-4140-8e8c-1ed176027eed None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:17.343446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f5c78cc4-e076-4140-8e8c-1ed176027eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:17.454869 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c78cc4-e076-4140-8e8c-1ed176027eed None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:17.462170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad0d1b-8415-4b37-a6f3-acfefdeabc47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586ddffa77a54521b42b2aa50f16cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:17.462845 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad0d1b-8415-4b37-a6f3-acfefdeabc47 service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:17.463126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad0d1b-8415-4b37-a6f3-acfefdeabc47 service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:17.463436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad0d1b-8415-4b37-a6f3-acfefdeabc47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:17.475469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad0d1b-8415-4b37-a6f3-acfefdeabc47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:17.477152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad0d1b-8415-4b37-a6f3-acfefdeabc47 service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:17.487379 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1099/2227] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jun 7 08:38:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:38:17.988364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f7724-0e7e-4e8d-9ac6-5fc3f5d625d4 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:17.991612 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8f7f7724-0e7e-4e8d-9ac6-5fc3f5d625d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:18.048852 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f7724-0e7e-4e8d-9ac6-5fc3f5d625d4 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:18.063063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:18.063979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:18.064270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:18.064551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] PATH_INFO: `/v3/users/6602673b1a9c49af9daeacc80df5be44` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:18.066430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6602673b1a9c49af9daeacc80df5be44)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:18.071485 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:18.401940 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6602673b1a9c49af9daeacc80df5be44'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:18.402664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6602673b1a9c49af9daeacc80df5be44'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:18.423155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6602673b1a9c49af9daeacc80df5be44'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:18.461990 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6602673b1a9c49af9daeacc80df5be44'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:18.469197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6602673b1a9c49af9daeacc80df5be44'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:18.497596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1129/2228] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:17 2022] DELETE /identity/v3/users/6602673b1a9c49af9daeacc80df5be44 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:18.502047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f6935-3bb1-408b-bd13-cea33308d3c9 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:18.502610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f6935-3bb1-408b-bd13-cea33308d3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:18.502748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f6935-3bb1-408b-bd13-cea33308d3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:18.538467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-419f6935-3bb1-408b-bd13-cea33308d3c9 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:18.539832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-419f6935-3bb1-408b-bd13-cea33308d3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:18.638368 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:18.656038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1100/2229] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:18.661558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:18.668712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:18.759573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518d79b-0795-445a-a1a9-7cd25163546f admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:18.764656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70911b9-6268-4373-987e-bfa0cacb7388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:18.765495 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70911b9-6268-4373-987e-bfa0cacb7388 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:18.765851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70911b9-6268-4373-987e-bfa0cacb7388 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:18.766353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70911b9-6268-4373-987e-bfa0cacb7388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:18.783857 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70911b9-6268-4373-987e-bfa0cacb7388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:18.787297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70911b9-6268-4373-987e-bfa0cacb7388 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:18.802798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1130/2230] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:38:19.390078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f6935-3bb1-408b-bd13-cea33308d3c9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:19.402647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-419f6935-3bb1-408b-bd13-cea33308d3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:19.466728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f6935-3bb1-408b-bd13-cea33308d3c9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:19.489317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe361e34-a601-4d68-9053-e11b6ab4804c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:19.490230 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe361e34-a601-4d68-9053-e11b6ab4804c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:19.490506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe361e34-a601-4d68-9053-e11b6ab4804c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:19.490975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe361e34-a601-4d68-9053-e11b6ab4804c admin admin] PATH_INFO: `/v3/projects/b6d278430459441db81df74d79a4aa20` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:19.493198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe361e34-a601-4d68-9053-e11b6ab4804c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6d278430459441db81df74d79a4aa20)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:19.496125 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe361e34-a601-4d68-9053-e11b6ab4804c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:19.613990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fe361e34-a601-4d68-9053-e11b6ab4804c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6d278430459441db81df74d79a4aa20', 'request_id': 'req-fe361e34-a601-4d68-9053-e11b6ab4804c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:19.633391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fe361e34-a601-4d68-9053-e11b6ab4804c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6d278430459441db81df74d79a4aa20', 'request_id': 'req-fe361e34-a601-4d68-9053-e11b6ab4804c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:19.655987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fe361e34-a601-4d68-9053-e11b6ab4804c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6d278430459441db81df74d79a4aa20', 'request_id': 'req-fe361e34-a601-4d68-9053-e11b6ab4804c'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:19.709915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fe361e34-a601-4d68-9053-e11b6ab4804c admin admin] The token cache is being invalidate because project b6d278430459441db81df74d79a4aa20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:19.713708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fe361e34-a601-4d68-9053-e11b6ab4804c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:19.726311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-fe361e34-a601-4d68-9053-e11b6ab4804c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:19.733751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1101/2231] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:19 2022] DELETE /identity/v3/projects/b6d278430459441db81df74d79a4aa20 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:19.743552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce62e96-45d1-4339-9908-eefc549d7351 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:19.750073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce62e96-45d1-4339-9908-eefc549d7351 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:19.751479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce62e96-45d1-4339-9908-eefc549d7351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:19.779967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-5ce62e96-45d1-4339-9908-eefc549d7351 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:19.783962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5ce62e96-45d1-4339-9908-eefc549d7351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:19.792998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:19.918312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1131/2232] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:19.992241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f0c35-0141-4b12-8f8c-28c15302f60c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:19.992653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f0c35-0141-4b12-8f8c-28c15302f60c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:19.993171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f0c35-0141-4b12-8f8c-28c15302f60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.021045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-d77f0c35-0141-4b12-8f8c-28c15302f60c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:20.022580 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d77f0c35-0141-4b12-8f8c-28c15302f60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:20.037979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1102/2233] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:20.039017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce62e96-45d1-4339-9908-eefc549d7351 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:20.053389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce62e96-45d1-4339-9908-eefc549d7351 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:20.062645 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90feacba-c3f1-43bc-aa21-ca028dbbc39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:20.063311 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90feacba-c3f1-43bc-aa21-ca028dbbc39e admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.063606 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90feacba-c3f1-43bc-aa21-ca028dbbc39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.063886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90feacba-c3f1-43bc-aa21-ca028dbbc39e admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.064640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90feacba-c3f1-43bc-aa21-ca028dbbc39e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:20.064950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90feacba-c3f1-43bc-aa21-ca028dbbc39e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:20.066911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90feacba-c3f1-43bc-aa21-ca028dbbc39e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:20.088972 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1132/2234] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:20.093441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f0c35-0141-4b12-8f8c-28c15302f60c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:20.105920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f0c35-0141-4b12-8f8c-28c15302f60c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:20.118678 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404f544-13ff-47e8-85d3-18c1af2aed3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:20.119383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404f544-13ff-47e8-85d3-18c1af2aed3a admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.119673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404f544-13ff-47e8-85d3-18c1af2aed3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.119948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404f544-13ff-47e8-85d3-18c1af2aed3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.121067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404f544-13ff-47e8-85d3-18c1af2aed3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:20.124185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404f544-13ff-47e8-85d3-18c1af2aed3a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:20.159584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1103/2235] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:20 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:20.164723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90feacba-c3f1-43bc-aa21-ca028dbbc39e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:20.179564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90feacba-c3f1-43bc-aa21-ca028dbbc39e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:20.273072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74075f36-c61b-446c-b251-3cba95218bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:20.274065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74075f36-c61b-446c-b251-3cba95218bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.274603 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74075f36-c61b-446c-b251-3cba95218bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.275591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74075f36-c61b-446c-b251-3cba95218bb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.277784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74075f36-c61b-446c-b251-3cba95218bb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:20.281825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74075f36-c61b-446c-b251-3cba95218bb0 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:20.316391 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1133/2236] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:20 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:20.320650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404f544-13ff-47e8-85d3-18c1af2aed3a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:20.339820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404f544-13ff-47e8-85d3-18c1af2aed3a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:20.354585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e6ad7-f324-4dc4-8717-71217608ae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:20.356253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e6ad7-f324-4dc4-8717-71217608ae25 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.356678 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e6ad7-f324-4dc4-8717-71217608ae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.356971 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e6ad7-f324-4dc4-8717-71217608ae25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.358014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e6ad7-f324-4dc4-8717-71217608ae25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:20.359756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e6ad7-f324-4dc4-8717-71217608ae25 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:20.378173 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1104/2237] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:20.385965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74075f36-c61b-446c-b251-3cba95218bb0 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:20.402384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74075f36-c61b-446c-b251-3cba95218bb0 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:20.409433 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86465cc-d277-4be3-932a-39095491d1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:20.410655 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86465cc-d277-4be3-932a-39095491d1af admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.411184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86465cc-d277-4be3-932a-39095491d1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.411725 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86465cc-d277-4be3-932a-39095491d1af admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.412999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86465cc-d277-4be3-932a-39095491d1af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:20.415065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86465cc-d277-4be3-932a-39095491d1af admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:20.447142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1134/2238] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:20.450163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e6ad7-f324-4dc4-8717-71217608ae25 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:20.483136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e6ad7-f324-4dc4-8717-71217608ae25 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:20.495252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db918c7-bb99-4276-9b9f-f29aac057bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:20.497192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db918c7-bb99-4276-9b9f-f29aac057bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.497708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db918c7-bb99-4276-9b9f-f29aac057bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.498215 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db918c7-bb99-4276-9b9f-f29aac057bd4 admin admin] PATH_INFO: `/v3/projects/60d9a41036b948d893c80fc128a729a4/users/afaeb0665aeb49cb9e55158b03812894/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.522824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db918c7-bb99-4276-9b9f-f29aac057bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d9a41036b948d893c80fc128a729a4, user_id=afaeb0665aeb49cb9e55158b03812894, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:20.526514 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db918c7-bb99-4276-9b9f-f29aac057bd4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:20.548035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1105/2239] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:20 2022] PUT /identity/v3/projects/60d9a41036b948d893c80fc128a729a4/users/afaeb0665aeb49cb9e55158b03812894/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:20.558977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde62771-7c78-441c-a5b7-95f76ae8f66d None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.559325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde62771-7c78-441c-a5b7-95f76ae8f66d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.559550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde62771-7c78-441c-a5b7-95f76ae8f66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.604504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-dde62771-7c78-441c-a5b7-95f76ae8f66d None None] MFA Rules not processed for user `afaeb0665aeb49cb9e55158b03812894`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:20.605883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-dde62771-7c78-441c-a5b7-95f76ae8f66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:20.790974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1135/2240] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:20.796822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db918c7-bb99-4276-9b9f-f29aac057bd4 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:20.959798 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db918c7-bb99-4276-9b9f-f29aac057bd4 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:20.977021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6b4f7-b868-45a0-ae69-d30a4517e1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:20.978582 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6b4f7-b868-45a0-ae69-d30a4517e1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.979136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6b4f7-b868-45a0-ae69-d30a4517e1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.979631 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6b4f7-b868-45a0-ae69-d30a4517e1a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.980606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6b4f7-b868-45a0-ae69-d30a4517e1a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:20.983037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6b4f7-b868-45a0-ae69-d30a4517e1a7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:21.038787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1106/2241] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:20 2022] POST /identity/v3/projects => generated 377 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:21.048376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde62771-7c78-441c-a5b7-95f76ae8f66d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:21.096434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde62771-7c78-441c-a5b7-95f76ae8f66d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:21.101593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:21.252304 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5dcf6-b7e6-461c-8fc5-cd4ac79a0a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:21.253443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5dcf6-b7e6-461c-8fc5-cd4ac79a0a1b admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:21.253765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5dcf6-b7e6-461c-8fc5-cd4ac79a0a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:21.254825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5dcf6-b7e6-461c-8fc5-cd4ac79a0a1b admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:21.255733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5dcf6-b7e6-461c-8fc5-cd4ac79a0a1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:21.257992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5dcf6-b7e6-461c-8fc5-cd4ac79a0a1b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:21.303370 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6b4f7-b868-45a0-ae69-d30a4517e1a7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:21.304289 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1136/2242] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:21 2022] POST /identity/v3/users => generated 289 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:21.327821 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:21.341716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6b4f7-b868-45a0-ae69-d30a4517e1a7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:21.353612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866ec91-7def-4d84-b2e7-70f3a3be26ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:21.354841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866ec91-7def-4d84-b2e7-70f3a3be26ca admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:21.355280 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866ec91-7def-4d84-b2e7-70f3a3be26ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:21.355713 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866ec91-7def-4d84-b2e7-70f3a3be26ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:21.356710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866ec91-7def-4d84-b2e7-70f3a3be26ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:21.359691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866ec91-7def-4d84-b2e7-70f3a3be26ca admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:21.378608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1107/2243] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:21.386448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5dcf6-b7e6-461c-8fc5-cd4ac79a0a1b admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:21.406695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5dcf6-b7e6-461c-8fc5-cd4ac79a0a1b admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:21.413921 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daba5d-e5cb-4daa-9591-9ab5cc18991f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:21.415002 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daba5d-e5cb-4daa-9591-9ab5cc18991f admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:21.415422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daba5d-e5cb-4daa-9591-9ab5cc18991f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:21.415881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daba5d-e5cb-4daa-9591-9ab5cc18991f admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:21.417109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02daba5d-e5cb-4daa-9591-9ab5cc18991f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:21.420041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02daba5d-e5cb-4daa-9591-9ab5cc18991f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:21.450236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1137/2244] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:21.450814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866ec91-7def-4d84-b2e7-70f3a3be26ca admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:21.484511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866ec91-7def-4d84-b2e7-70f3a3be26ca admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:21.492238 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88b786-8ad3-407c-a1e0-68e71e38fd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:21.493296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88b786-8ad3-407c-a1e0-68e71e38fd45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:21.493657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88b786-8ad3-407c-a1e0-68e71e38fd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:21.493975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88b786-8ad3-407c-a1e0-68e71e38fd45 admin admin] PATH_INFO: `/v3/projects/30f558420d1341ebb8a0de75b852251d/users/8679151f93444f818774760c4632697d/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:21.527409 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88b786-8ad3-407c-a1e0-68e71e38fd45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f558420d1341ebb8a0de75b852251d, user_id=8679151f93444f818774760c4632697d, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:21.532220 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88b786-8ad3-407c-a1e0-68e71e38fd45 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:21.559641 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1108/2245] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:21 2022] PUT /identity/v3/projects/30f558420d1341ebb8a0de75b852251d/users/8679151f93444f818774760c4632697d/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:21.578779 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa00a87-5fd0-40fc-9d7b-d0c053e9b34c None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:21.579274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa00a87-5fd0-40fc-9d7b-d0c053e9b34c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:21.579759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa00a87-5fd0-40fc-9d7b-d0c053e9b34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:21.626982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-0fa00a87-5fd0-40fc-9d7b-d0c053e9b34c None None] MFA Rules not processed for user `8679151f93444f818774760c4632697d`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:21.628470 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0fa00a87-5fd0-40fc-9d7b-d0c053e9b34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:21.750640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1138/2246] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:21.756001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88b786-8ad3-407c-a1e0-68e71e38fd45 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:21.910011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88b786-8ad3-407c-a1e0-68e71e38fd45 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:21.914996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27730d7e-652e-40ce-b728-69ad2b89d172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:21.915661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27730d7e-652e-40ce-b728-69ad2b89d172 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:21.915951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27730d7e-652e-40ce-b728-69ad2b89d172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:21.916246 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27730d7e-652e-40ce-b728-69ad2b89d172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:21.917078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27730d7e-652e-40ce-b728-69ad2b89d172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:21.919013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27730d7e-652e-40ce-b728-69ad2b89d172 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:21.958873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1109/2247] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:21 2022] POST /identity/v3/projects => generated 377 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:21.963268 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa00a87-5fd0-40fc-9d7b-d0c053e9b34c None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:21.982497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa00a87-5fd0-40fc-9d7b-d0c053e9b34c None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:22.110295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee5949-a086-4ddb-8a0b-b67a79d3456c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:22.111318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee5949-a086-4ddb-8a0b-b67a79d3456c admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:22.111746 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee5949-a086-4ddb-8a0b-b67a79d3456c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:22.112183 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee5949-a086-4ddb-8a0b-b67a79d3456c admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:22.113028 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee5949-a086-4ddb-8a0b-b67a79d3456c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:22.116175 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee5949-a086-4ddb-8a0b-b67a79d3456c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:22.151484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1139/2248] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:21 2022] POST /identity/v3/users => generated 295 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:22.153011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27730d7e-652e-40ce-b728-69ad2b89d172 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:22.166458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27730d7e-652e-40ce-b728-69ad2b89d172 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:22.171855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e46e4-4e38-4aed-9a85-eb4f56f6d221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:22.172771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e46e4-4e38-4aed-9a85-eb4f56f6d221 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:22.173057 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e46e4-4e38-4aed-9a85-eb4f56f6d221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:22.173365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e46e4-4e38-4aed-9a85-eb4f56f6d221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:22.174233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e46e4-4e38-4aed-9a85-eb4f56f6d221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:22.175851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e46e4-4e38-4aed-9a85-eb4f56f6d221 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:22.188602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1110/2249] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:22.193646 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee5949-a086-4ddb-8a0b-b67a79d3456c admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:22.201544 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:22.205788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee5949-a086-4ddb-8a0b-b67a79d3456c admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:22.212037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c56261-e038-4957-ba72-f69a78ff385d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:22.212946 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c56261-e038-4957-ba72-f69a78ff385d admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:22.213260 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c56261-e038-4957-ba72-f69a78ff385d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:22.213540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c56261-e038-4957-ba72-f69a78ff385d admin admin] PATH_INFO: `/v3/projects/19b47a318f074f69a727ae63d52b5d1a/users/4e7d9e5a721142308efc0c4f6d05c25d/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:22.232933 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c56261-e038-4957-ba72-f69a78ff385d admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b47a318f074f69a727ae63d52b5d1a, user_id=4e7d9e5a721142308efc0c4f6d05c25d, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:22.235738 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c56261-e038-4957-ba72-f69a78ff385d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:22.259968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1140/2250] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:22 2022] PUT /identity/v3/projects/19b47a318f074f69a727ae63d52b5d1a/users/4e7d9e5a721142308efc0c4f6d05c25d/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:22.263871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e46e4-4e38-4aed-9a85-eb4f56f6d221 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:22.274977 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e46e4-4e38-4aed-9a85-eb4f56f6d221 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:22.368199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe4081-c79a-4e3a-b1d2-2d96b3f1377a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:22.368944 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe4081-c79a-4e3a-b1d2-2d96b3f1377a admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:22.369267 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe4081-c79a-4e3a-b1d2-2d96b3f1377a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:22.369613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe4081-c79a-4e3a-b1d2-2d96b3f1377a admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:22.370371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe4081-c79a-4e3a-b1d2-2d96b3f1377a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:22.372255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe4081-c79a-4e3a-b1d2-2d96b3f1377a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:22.392205 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1111/2251] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:22.393902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c56261-e038-4957-ba72-f69a78ff385d admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:22.434223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c56261-e038-4957-ba72-f69a78ff385d admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:22.470634 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4eeab-52c2-4993-a15c-e773ff75e4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:22.471813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4eeab-52c2-4993-a15c-e773ff75e4bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:22.472399 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4eeab-52c2-4993-a15c-e773ff75e4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:22.473292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4eeab-52c2-4993-a15c-e773ff75e4bd admin admin] PATH_INFO: `/v3/projects/19b47a318f074f69a727ae63d52b5d1a/users/4e7d9e5a721142308efc0c4f6d05c25d/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:22.475462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4eeab-52c2-4993-a15c-e773ff75e4bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b47a318f074f69a727ae63d52b5d1a, user_id=4e7d9e5a721142308efc0c4f6d05c25d, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:22.478258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4eeab-52c2-4993-a15c-e773ff75e4bd admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:22.490395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1141/2252] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:22 2022] PUT /identity/v3/projects/19b47a318f074f69a727ae63d52b5d1a/users/4e7d9e5a721142308efc0c4f6d05c25d/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:22.502958 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd91030-9abb-4c9a-82db-031648d091bd None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:22.503357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd91030-9abb-4c9a-82db-031648d091bd None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:22.504144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd91030-9abb-4c9a-82db-031648d091bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:22.548351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-cdd91030-9abb-4c9a-82db-031648d091bd None None] MFA Rules not processed for user `4e7d9e5a721142308efc0c4f6d05c25d`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:22.552372 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cdd91030-9abb-4c9a-82db-031648d091bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:22.587893 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:22.639717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1112/2253] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:22.652092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4eeab-52c2-4993-a15c-e773ff75e4bd admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:22.653973 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-64a4eeab-52c2-4993-a15c-e773ff75e4bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:22.781195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4eeab-52c2-4993-a15c-e773ff75e4bd admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:22.785768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58a093-8cd1-4544-a6df-a171ae267492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:22.786540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58a093-8cd1-4544-a6df-a171ae267492 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:22.786992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58a093-8cd1-4544-a6df-a171ae267492 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:22.787306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58a093-8cd1-4544-a6df-a171ae267492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:22.862768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58a093-8cd1-4544-a6df-a171ae267492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:22.865413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58a093-8cd1-4544-a6df-a171ae267492 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:22.878459 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1142/2254] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:24.657928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd91030-9abb-4c9a-82db-031648d091bd None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:24.661787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cdd91030-9abb-4c9a-82db-031648d091bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:24.678708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd91030-9abb-4c9a-82db-031648d091bd None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:24.685214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7408ed8-3b2f-4923-98e2-e2dcf6408e14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:24.686183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7408ed8-3b2f-4923-98e2-e2dcf6408e14 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:24.686878 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7408ed8-3b2f-4923-98e2-e2dcf6408e14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:24.687328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7408ed8-3b2f-4923-98e2-e2dcf6408e14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:24.770951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7408ed8-3b2f-4923-98e2-e2dcf6408e14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:24.772435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7408ed8-3b2f-4923-98e2-e2dcf6408e14 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:24.797438 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1113/2255] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:26.133567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58a093-8cd1-4544-a6df-a171ae267492 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:26.168246 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58a093-8cd1-4544-a6df-a171ae267492 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:26.182424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1b898-e397-478c-9148-d043548d6fdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:26.183201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1b898-e397-478c-9148-d043548d6fdc service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:26.183494 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1b898-e397-478c-9148-d043548d6fdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:26.183798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1b898-e397-478c-9148-d043548d6fdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:26.200709 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1b898-e397-478c-9148-d043548d6fdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:26.212656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1b898-e397-478c-9148-d043548d6fdc service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:26.259172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1143/2256] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:28.342702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aad31-6183-4c8d-93ec-616919cedae4 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:28.343295 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aad31-6183-4c8d-93ec-616919cedae4 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:28.343295 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aad31-6183-4c8d-93ec-616919cedae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:28.362675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a15aad31-6183-4c8d-93ec-616919cedae4 None None] MFA Rules not processed for user `4f603b289d964ee3ba3931948acf6788`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:28.365139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a15aad31-6183-4c8d-93ec-616919cedae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:28.434224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1114/2257] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:31.011016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1b898-e397-478c-9148-d043548d6fdc service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:31.082456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:31.101211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1b898-e397-478c-9148-d043548d6fdc service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:31.118852 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:31.119778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:31.120065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:31.120606 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] PATH_INFO: `/v3/users/afaeb0665aeb49cb9e55158b03812894` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:31.122197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] RBAC: Authorizing `identity:delete_user(user_id=afaeb0665aeb49cb9e55158b03812894)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:31.124447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:31.216868 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afaeb0665aeb49cb9e55158b03812894'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:31.217753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afaeb0665aeb49cb9e55158b03812894'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:31.233360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afaeb0665aeb49cb9e55158b03812894'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:31.253368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afaeb0665aeb49cb9e55158b03812894'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:31.254948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afaeb0665aeb49cb9e55158b03812894'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:31.277128 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1144/2258] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:31 2022] DELETE /identity/v3/users/afaeb0665aeb49cb9e55158b03812894 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:31.279052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d05ff-dae7-4e79-a430-b41a359ffad8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:31.279607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d05ff-dae7-4e79-a430-b41a359ffad8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:31.279607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d05ff-dae7-4e79-a430-b41a359ffad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:31.303530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-405d05ff-dae7-4e79-a430-b41a359ffad8 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:31.304966 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-405d05ff-dae7-4e79-a430-b41a359ffad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:31.477727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1115/2259] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:31.489812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:31.492759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:31.598079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40043a70-ee37-4fb7-936e-208a34429adc admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:31.603627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8455a65-b3ae-49f3-86f7-9c3c769f8354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:31.604223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8455a65-b3ae-49f3-86f7-9c3c769f8354 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:31.604664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8455a65-b3ae-49f3-86f7-9c3c769f8354 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:31.604949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8455a65-b3ae-49f3-86f7-9c3c769f8354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:31.616719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8455a65-b3ae-49f3-86f7-9c3c769f8354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:31.619032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8455a65-b3ae-49f3-86f7-9c3c769f8354 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:31.634220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1145/2260] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:32.238993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d05ff-dae7-4e79-a430-b41a359ffad8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:32.242456 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-405d05ff-dae7-4e79-a430-b41a359ffad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:32.265739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d05ff-dae7-4e79-a430-b41a359ffad8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:32.271335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:32.272434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:32.272563 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:32.272819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] PATH_INFO: `/v3/projects/60d9a41036b948d893c80fc128a729a4` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:32.274703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60d9a41036b948d893c80fc128a729a4)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:32.276665 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:32.281560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:32.381144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60d9a41036b948d893c80fc128a729a4', 'request_id': 'req-5e155900-7de9-4b4d-abdf-446ea75e2909'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:32.390776 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60d9a41036b948d893c80fc128a729a4', 'request_id': 'req-5e155900-7de9-4b4d-abdf-446ea75e2909'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:32.402503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60d9a41036b948d893c80fc128a729a4', 'request_id': 'req-5e155900-7de9-4b4d-abdf-446ea75e2909'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:32.423216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] The token cache is being invalidate because project 60d9a41036b948d893c80fc128a729a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:32.423520 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:32.424244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:32.429132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1116/2261] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:32 2022] DELETE /identity/v3/projects/60d9a41036b948d893c80fc128a729a4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:32.432953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8455a65-b3ae-49f3-86f7-9c3c769f8354 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:32.451229 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8455a65-b3ae-49f3-86f7-9c3c769f8354 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:32.563691 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:32.564970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:32.565411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:32.565676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] PATH_INFO: `/v3/users/8679151f93444f818774760c4632697d` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:32.568140 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8679151f93444f818774760c4632697d)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:32.570628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:32.592654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:32.654408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8679151f93444f818774760c4632697d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:32.655848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8679151f93444f818774760c4632697d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:32.679376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8679151f93444f818774760c4632697d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:32.699490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8679151f93444f818774760c4632697d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:32.700262 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8679151f93444f818774760c4632697d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:32.720611 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1146/2262] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:32 2022] DELETE /identity/v3/users/8679151f93444f818774760c4632697d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:33.398559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:33.400441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:33.428128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e155900-7de9-4b4d-abdf-446ea75e2909 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:33.589918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:33.591656 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:33.599476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:33.603888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] PATH_INFO: `/v3/projects/30f558420d1341ebb8a0de75b852251d` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:33.606387 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f558420d1341ebb8a0de75b852251d)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:33.609903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:33.715436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f558420d1341ebb8a0de75b852251d', 'request_id': 'req-6c391484-dffc-414d-989b-d82e6017bbcc'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:33.722200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f558420d1341ebb8a0de75b852251d', 'request_id': 'req-6c391484-dffc-414d-989b-d82e6017bbcc'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:33.736970 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f558420d1341ebb8a0de75b852251d', 'request_id': 'req-6c391484-dffc-414d-989b-d82e6017bbcc'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:33.762727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] The token cache is being invalidate because project 30f558420d1341ebb8a0de75b852251d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:33.762727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:33.763750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:33.766191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1117/2263] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:33 2022] DELETE /identity/v3/projects/30f558420d1341ebb8a0de75b852251d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:33.771772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:33.772923 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:33.790544 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2d805-0c92-4035-896b-75eb64a3df2c admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:33.881888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e902b-3fc5-4acd-91b0-8f22cc230da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:33.882772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e902b-3fc5-4acd-91b0-8f22cc230da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:33.883058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e902b-3fc5-4acd-91b0-8f22cc230da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:33.883337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e902b-3fc5-4acd-91b0-8f22cc230da9 admin admin] PATH_INFO: `/v3/users/4e7d9e5a721142308efc0c4f6d05c25d` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:33.889386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e902b-3fc5-4acd-91b0-8f22cc230da9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e7d9e5a721142308efc0c4f6d05c25d)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:33.889386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e902b-3fc5-4acd-91b0-8f22cc230da9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:33.990676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a22e902b-3fc5-4acd-91b0-8f22cc230da9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e7d9e5a721142308efc0c4f6d05c25d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:33.991347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a22e902b-3fc5-4acd-91b0-8f22cc230da9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e7d9e5a721142308efc0c4f6d05c25d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:33.999938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a22e902b-3fc5-4acd-91b0-8f22cc230da9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e7d9e5a721142308efc0c4f6d05c25d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:34.023033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a22e902b-3fc5-4acd-91b0-8f22cc230da9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e7d9e5a721142308efc0c4f6d05c25d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:34.023865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a22e902b-3fc5-4acd-91b0-8f22cc230da9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e7d9e5a721142308efc0c4f6d05c25d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:34.043841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1147/2264] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:33 2022] DELETE /identity/v3/users/4e7d9e5a721142308efc0c4f6d05c25d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:35.280211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:35.283191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:35.317564 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c391484-dffc-414d-989b-d82e6017bbcc admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:35.370990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:35.500707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:35.503236 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:35.504626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:35.505649 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6 admin admin] PATH_INFO: `/v3/projects/19b47a318f074f69a727ae63d52b5d1a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:35.511833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b47a318f074f69a727ae63d52b5d1a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:35.516128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:35.613142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b47a318f074f69a727ae63d52b5d1a', 'request_id': 'req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:35.623914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b47a318f074f69a727ae63d52b5d1a', 'request_id': 'req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:35.642869 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b47a318f074f69a727ae63d52b5d1a', 'request_id': 'req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:35.671722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6 admin admin] The token cache is being invalidate because project 19b47a318f074f69a727ae63d52b5d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:35.672017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:35.672797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cd371e52-22bb-4509-adb9-0e8d8a42c9a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:35.676616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1118/2265] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:35 2022] DELETE /identity/v3/projects/19b47a318f074f69a727ae63d52b5d1a => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:35.684295 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b968c-5361-4e98-8b72-73ed0de62b40 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:35.684918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b968c-5361-4e98-8b72-73ed0de62b40 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:35.684918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b968c-5361-4e98-8b72-73ed0de62b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:35.724084 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-726b968c-5361-4e98-8b72-73ed0de62b40 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:35.725849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-726b968c-5361-4e98-8b72-73ed0de62b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:35.740704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:35.939429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1148/2266] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:35.947905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c5cd0-a580-4390-964f-4d28a9eb9bfc None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:35.948442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c5cd0-a580-4390-964f-4d28a9eb9bfc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:35.948812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c5cd0-a580-4390-964f-4d28a9eb9bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:35.982251 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-189c5cd0-a580-4390-964f-4d28a9eb9bfc None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:35.983816 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-189c5cd0-a580-4390-964f-4d28a9eb9bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:36.014239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1119/2267] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:36.018880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b968c-5361-4e98-8b72-73ed0de62b40 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:36.059422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b968c-5361-4e98-8b72-73ed0de62b40 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:36.076477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0edac-0d91-494b-86a9-cdd78f6993b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:36.080646 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0edac-0d91-494b-86a9-cdd78f6993b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:36.084522 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0edac-0d91-494b-86a9-cdd78f6993b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:36.085259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0edac-0d91-494b-86a9-cdd78f6993b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:36.086264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d0edac-0d91-494b-86a9-cdd78f6993b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:36.086769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d0edac-0d91-494b-86a9-cdd78f6993b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:36.090171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d0edac-0d91-494b-86a9-cdd78f6993b3 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:36.142292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1149/2268] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:36.145733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c5cd0-a580-4390-964f-4d28a9eb9bfc None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:36.163993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c5cd0-a580-4390-964f-4d28a9eb9bfc None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:36.177498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345bc21-24ec-49f8-9025-4b51294a1e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:36.179480 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345bc21-24ec-49f8-9025-4b51294a1e4c admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:36.179786 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345bc21-24ec-49f8-9025-4b51294a1e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:36.180081 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345bc21-24ec-49f8-9025-4b51294a1e4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:36.185664 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345bc21-24ec-49f8-9025-4b51294a1e4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:36.187631 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345bc21-24ec-49f8-9025-4b51294a1e4c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:36.256106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1120/2269] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:36 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:36.260333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0edac-0d91-494b-86a9-cdd78f6993b3 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:36.275845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0edac-0d91-494b-86a9-cdd78f6993b3 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:36.410614 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb15590-ddcb-46a3-b19a-805b10a2d8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:36.411329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb15590-ddcb-46a3-b19a-805b10a2d8db admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:36.411621 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb15590-ddcb-46a3-b19a-805b10a2d8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:36.411894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb15590-ddcb-46a3-b19a-805b10a2d8db admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:36.412732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb15590-ddcb-46a3-b19a-805b10a2d8db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:36.414742 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb15590-ddcb-46a3-b19a-805b10a2d8db admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:36.469818 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345bc21-24ec-49f8-9025-4b51294a1e4c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:36.473855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1150/2270] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:36 2022] POST /identity/v3/users => generated 285 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:36.487889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345bc21-24ec-49f8-9025-4b51294a1e4c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:36.502145 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68578668-acea-4888-bae5-365516776f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:36.503012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68578668-acea-4888-bae5-365516776f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:36.503457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68578668-acea-4888-bae5-365516776f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:36.504789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68578668-acea-4888-bae5-365516776f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:36.505156 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68578668-acea-4888-bae5-365516776f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:36.507080 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68578668-acea-4888-bae5-365516776f7b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:36.530937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1121/2271] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:38:36.534907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb15590-ddcb-46a3-b19a-805b10a2d8db admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:36.553956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb15590-ddcb-46a3-b19a-805b10a2d8db admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:36.568192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886c27d-b582-408e-957a-5ffe1b46822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:36.569519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886c27d-b582-408e-957a-5ffe1b46822f admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:36.570154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886c27d-b582-408e-957a-5ffe1b46822f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:36.570738 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886c27d-b582-408e-957a-5ffe1b46822f admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:36.573206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886c27d-b582-408e-957a-5ffe1b46822f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:36.575693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886c27d-b582-408e-957a-5ffe1b46822f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:36.603596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1151/2272] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:36.617491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68578668-acea-4888-bae5-365516776f7b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:36.631384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68578668-acea-4888-bae5-365516776f7b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:36.642812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4db9f-a599-4328-8278-e3e9cff295c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:36.643734 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4db9f-a599-4328-8278-e3e9cff295c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:36.644013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4db9f-a599-4328-8278-e3e9cff295c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:36.644944 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4db9f-a599-4328-8278-e3e9cff295c0 admin admin] PATH_INFO: `/v3/projects/e891f44f3e7f4abdbaa4cf938f315a15/users/8ab6821a01c8465ab37e3304f1e26f6f/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:36.694473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4db9f-a599-4328-8278-e3e9cff295c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e891f44f3e7f4abdbaa4cf938f315a15, user_id=8ab6821a01c8465ab37e3304f1e26f6f, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:36.696782 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4db9f-a599-4328-8278-e3e9cff295c0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:36.721567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1122/2273] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:36 2022] PUT /identity/v3/projects/e891f44f3e7f4abdbaa4cf938f315a15/users/8ab6821a01c8465ab37e3304f1e26f6f/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:36.733974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a06e50-303e-44e4-af4b-b618bae66264 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:36.734524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a06e50-303e-44e4-af4b-b618bae66264 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:36.734750 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a06e50-303e-44e4-af4b-b618bae66264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:36.794276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-b7a06e50-303e-44e4-af4b-b618bae66264 None None] MFA Rules not processed for user `8ab6821a01c8465ab37e3304f1e26f6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:36.795528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b7a06e50-303e-44e4-af4b-b618bae66264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:36.951456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1152/2274] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:38:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:36.973639 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4db9f-a599-4328-8278-e3e9cff295c0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:36.976675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5df4db9f-a599-4328-8278-e3e9cff295c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:37.057284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:37.104263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4db9f-a599-4328-8278-e3e9cff295c0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:37.108510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a2a40-55de-417c-9cce-f2f3bcfa32c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e4531ac696445ba5bfa9870984befd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:37.109208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a2a40-55de-417c-9cce-f2f3bcfa32c2 service glance] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:37.109496 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a2a40-55de-417c-9cce-f2f3bcfa32c2 service glance] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:37.109775 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a2a40-55de-417c-9cce-f2f3bcfa32c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:37.123408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523a2a40-55de-417c-9cce-f2f3bcfa32c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:37.124915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523a2a40-55de-417c-9cce-f2f3bcfa32c2 service glance] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:37.142075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1123/2275] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:36 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:38:46.945532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:47.134450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=14 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:59.765828 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928afa2-f6a8-4bfb-aadb-07488c170fcb None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:59.766334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928afa2-f6a8-4bfb-aadb-07488c170fcb None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:59.766427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928afa2-f6a8-4bfb-aadb-07488c170fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:59.813510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-5928afa2-f6a8-4bfb-aadb-07488c170fcb None None] MFA Rules not processed for user `26d6eb06b23b4db88261ae58d650736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:59.814780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5928afa2-f6a8-4bfb-aadb-07488c170fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:59.880065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1153/2276] 10.222.0.39 () {58 vars in 967 bytes} [Tue Jun 7 08:38:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:59.894054 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a2a40-55de-417c-9cce-f2f3bcfa32c2 service glance] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:59.895525 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-523a2a40-55de-417c-9cce-f2f3bcfa32c2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:59.993042 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a2a40-55de-417c-9cce-f2f3bcfa32c2 service glance] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:59.999645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef13f58-174b-4839-9f11-24a9778ddead service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586ddffa77a54521b42b2aa50f16cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:00.000748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef13f58-174b-4839-9f11-24a9778ddead service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:00.000876 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef13f58-174b-4839-9f11-24a9778ddead service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:00.001187 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef13f58-174b-4839-9f11-24a9778ddead service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:00.013602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef13f58-174b-4839-9f11-24a9778ddead service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:00.016311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef13f58-174b-4839-9f11-24a9778ddead service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:00.025729 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1124/2277] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jun 7 08:38:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:39:02.101835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928afa2-f6a8-4bfb-aadb-07488c170fcb None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.103950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5928afa2-f6a8-4bfb-aadb-07488c170fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:02.193736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928afa2-f6a8-4bfb-aadb-07488c170fcb None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.198529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5745c39-5207-4e90-b357-83353fecfe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.199419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5745c39-5207-4e90-b357-83353fecfe7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.199695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5745c39-5207-4e90-b357-83353fecfe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.199969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5745c39-5207-4e90-b357-83353fecfe7b admin admin] PATH_INFO: `/v3/users/ffa54c295b764c07ba5d4c1d911fe7ce` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.201220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5745c39-5207-4e90-b357-83353fecfe7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffa54c295b764c07ba5d4c1d911fe7ce)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.203153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5745c39-5207-4e90-b357-83353fecfe7b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.252791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c5745c39-5207-4e90-b357-83353fecfe7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffa54c295b764c07ba5d4c1d911fe7ce'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.253468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c5745c39-5207-4e90-b357-83353fecfe7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffa54c295b764c07ba5d4c1d911fe7ce'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.260556 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c5745c39-5207-4e90-b357-83353fecfe7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffa54c295b764c07ba5d4c1d911fe7ce'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.272585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c5745c39-5207-4e90-b357-83353fecfe7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffa54c295b764c07ba5d4c1d911fe7ce'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.273285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-c5745c39-5207-4e90-b357-83353fecfe7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffa54c295b764c07ba5d4c1d911fe7ce'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.274945 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:02.287413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1154/2278] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:02 2022] DELETE /identity/v3/users/ffa54c295b764c07ba5d4c1d911fe7ce => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:02.466330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef13f58-174b-4839-9f11-24a9778ddead service swift] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.477068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef13f58-174b-4839-9f11-24a9778ddead service swift] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.550219 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4d556-d862-4726-84c7-9d246fd42137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.551206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4d556-d862-4726-84c7-9d246fd42137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.551484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4d556-d862-4726-84c7-9d246fd42137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.551769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4d556-d862-4726-84c7-9d246fd42137 admin admin] PATH_INFO: `/v3/projects/2ebcc3d147b546299812829603b173d2` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.553067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4d556-d862-4726-84c7-9d246fd42137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ebcc3d147b546299812829603b173d2)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.555094 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4d556-d862-4726-84c7-9d246fd42137 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.605702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dd4d556-d862-4726-84c7-9d246fd42137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ebcc3d147b546299812829603b173d2', 'request_id': 'req-0dd4d556-d862-4726-84c7-9d246fd42137'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.611504 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dd4d556-d862-4726-84c7-9d246fd42137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ebcc3d147b546299812829603b173d2', 'request_id': 'req-0dd4d556-d862-4726-84c7-9d246fd42137'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.619256 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dd4d556-d862-4726-84c7-9d246fd42137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ebcc3d147b546299812829603b173d2', 'request_id': 'req-0dd4d556-d862-4726-84c7-9d246fd42137'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.627132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:02.632082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dd4d556-d862-4726-84c7-9d246fd42137 admin admin] The token cache is being invalidate because project 2ebcc3d147b546299812829603b173d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:02.632750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dd4d556-d862-4726-84c7-9d246fd42137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.633634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dd4d556-d862-4726-84c7-9d246fd42137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.636163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1125/2279] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:02 2022] DELETE /identity/v3/projects/2ebcc3d147b546299812829603b173d2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:02.641168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e9c82-6de6-4151-9011-348a3a808134 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.641491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e9c82-6de6-4151-9011-348a3a808134 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.641836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e9c82-6de6-4151-9011-348a3a808134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.660601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-f53e9c82-6de6-4151-9011-348a3a808134 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:02.661835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f53e9c82-6de6-4151-9011-348a3a808134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:02.722768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1155/2280] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:02.730904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ebf11-0489-4e9b-b96e-9dd00c8c27da None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.731201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ebf11-0489-4e9b-b96e-9dd00c8c27da None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.731488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ebf11-0489-4e9b-b96e-9dd00c8c27da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.750013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c57ebf11-0489-4e9b-b96e-9dd00c8c27da None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:02.751344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c57ebf11-0489-4e9b-b96e-9dd00c8c27da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:02.761127 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1126/2281] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:02.766095 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e9c82-6de6-4151-9011-348a3a808134 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.777201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e9c82-6de6-4151-9011-348a3a808134 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.781842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93314-865d-4dc9-aa06-394b57c8292e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.782496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93314-865d-4dc9-aa06-394b57c8292e admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.782778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93314-865d-4dc9-aa06-394b57c8292e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.783052 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93314-865d-4dc9-aa06-394b57c8292e admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.783771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae93314-865d-4dc9-aa06-394b57c8292e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:02.784073 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae93314-865d-4dc9-aa06-394b57c8292e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.785700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae93314-865d-4dc9-aa06-394b57c8292e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.797410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1156/2282] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:02.801705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ebf11-0489-4e9b-b96e-9dd00c8c27da None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.811289 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ebf11-0489-4e9b-b96e-9dd00c8c27da None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.815980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598db075-8c61-485c-abfc-6903bfad708f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.816634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598db075-8c61-485c-abfc-6903bfad708f admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.816920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598db075-8c61-485c-abfc-6903bfad708f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.817240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598db075-8c61-485c-abfc-6903bfad708f admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.817945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598db075-8c61-485c-abfc-6903bfad708f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.819840 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598db075-8c61-485c-abfc-6903bfad708f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.848345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1127/2283] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:02 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:02.852562 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93314-865d-4dc9-aa06-394b57c8292e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.863200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93314-865d-4dc9-aa06-394b57c8292e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.924957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efdf14-2667-4cf3-9ded-ede3ffc5f765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.925671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efdf14-2667-4cf3-9ded-ede3ffc5f765 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.925967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efdf14-2667-4cf3-9ded-ede3ffc5f765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.926249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efdf14-2667-4cf3-9ded-ede3ffc5f765 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.926936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efdf14-2667-4cf3-9ded-ede3ffc5f765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.928905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efdf14-2667-4cf3-9ded-ede3ffc5f765 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.950673 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1157/2284] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:02 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:02.954460 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598db075-8c61-485c-abfc-6903bfad708f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.965136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598db075-8c61-485c-abfc-6903bfad708f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.969931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238f055-8f38-430f-abdc-1dd879bbfc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.970632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238f055-8f38-430f-abdc-1dd879bbfc58 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.970919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238f055-8f38-430f-abdc-1dd879bbfc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.971199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238f055-8f38-430f-abdc-1dd879bbfc58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.971936 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238f055-8f38-430f-abdc-1dd879bbfc58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.973606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238f055-8f38-430f-abdc-1dd879bbfc58 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.986167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1128/2285] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:02.989880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efdf14-2667-4cf3-9ded-ede3ffc5f765 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:03.002355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efdf14-2667-4cf3-9ded-ede3ffc5f765 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:03.007807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada273a3-46bf-4505-8d3d-aee254d37e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:03.008745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada273a3-46bf-4505-8d3d-aee254d37e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.009194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada273a3-46bf-4505-8d3d-aee254d37e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.010268 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada273a3-46bf-4505-8d3d-aee254d37e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.011126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada273a3-46bf-4505-8d3d-aee254d37e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:03.013172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada273a3-46bf-4505-8d3d-aee254d37e0d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:03.029129 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1158/2286] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:03.031043 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238f055-8f38-430f-abdc-1dd879bbfc58 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:03.041507 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238f055-8f38-430f-abdc-1dd879bbfc58 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:03.046091 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fb056-d392-40de-a22f-0bec8d4f0bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:03.046967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fb056-d392-40de-a22f-0bec8d4f0bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.047249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fb056-d392-40de-a22f-0bec8d4f0bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.047551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fb056-d392-40de-a22f-0bec8d4f0bcd admin admin] PATH_INFO: `/v3/projects/9a574283f4d74e389163486ccc060d81/users/4a27782a48ec45f8939f5015af3dbaa9/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.063056 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fb056-d392-40de-a22f-0bec8d4f0bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a574283f4d74e389163486ccc060d81, user_id=4a27782a48ec45f8939f5015af3dbaa9, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:03.065762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fb056-d392-40de-a22f-0bec8d4f0bcd admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:03.078984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1129/2287] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:03 2022] PUT /identity/v3/projects/9a574283f4d74e389163486ccc060d81/users/4a27782a48ec45f8939f5015af3dbaa9/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:03.089638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b460f7e-31d0-4723-b095-bd04d53d601c None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.089932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b460f7e-31d0-4723-b095-bd04d53d601c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.090215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b460f7e-31d0-4723-b095-bd04d53d601c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.119999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-5b460f7e-31d0-4723-b095-bd04d53d601c None None] MFA Rules not processed for user `4a27782a48ec45f8939f5015af3dbaa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:03.121266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5b460f7e-31d0-4723-b095-bd04d53d601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:03.164979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:03.188528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1159/2288] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:39:03 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:03.201835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fb056-d392-40de-a22f-0bec8d4f0bcd admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:03.203655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a28fb056-d392-40de-a22f-0bec8d4f0bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:03.263904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fb056-d392-40de-a22f-0bec8d4f0bcd admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:03.267632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8cd99-66bb-49bd-8a62-aa35af46473b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e4531ac696445ba5bfa9870984befd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:03.268307 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8cd99-66bb-49bd-8a62-aa35af46473b service glance] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.268591 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8cd99-66bb-49bd-8a62-aa35af46473b service glance] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.268868 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8cd99-66bb-49bd-8a62-aa35af46473b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.283371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8cd99-66bb-49bd-8a62-aa35af46473b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:03.285600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8cd99-66bb-49bd-8a62-aa35af46473b service glance] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:03.296604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1130/2289] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:03 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:13.185413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:13.292225 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:37.694638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a534061-297b-48d2-99ff-3e788e4e0339 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:37.695165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a534061-297b-48d2-99ff-3e788e4e0339 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:37.695287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a534061-297b-48d2-99ff-3e788e4e0339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:37.720820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-8a534061-297b-48d2-99ff-3e788e4e0339 None None] MFA Rules not processed for user `26d6eb06b23b4db88261ae58d650736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:37.722993 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8a534061-297b-48d2-99ff-3e788e4e0339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:37.811237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1160/2290] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jun 7 08:39:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:37.839151 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8cd99-66bb-49bd-8a62-aa35af46473b service glance] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:37.840711 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b8a8cd99-66bb-49bd-8a62-aa35af46473b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:37.979087 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8cd99-66bb-49bd-8a62-aa35af46473b service glance] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:37.986454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82677536-e787-47c4-9e3c-3aef8606e3f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586ddffa77a54521b42b2aa50f16cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:37.987096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82677536-e787-47c4-9e3c-3aef8606e3f2 service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:37.987394 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82677536-e787-47c4-9e3c-3aef8606e3f2 service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:37.987675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82677536-e787-47c4-9e3c-3aef8606e3f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:37.998325 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82677536-e787-47c4-9e3c-3aef8606e3f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:37.999988 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82677536-e787-47c4-9e3c-3aef8606e3f2 service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:38.009855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1131/2291] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jun 7 08:39:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 08:39:47.803075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=25 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:48.006170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=32 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:54.720863 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a534061-297b-48d2-99ff-3e788e4e0339 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:54.722568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8a534061-297b-48d2-99ff-3e788e4e0339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:54.778345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82677536-e787-47c4-9e3c-3aef8606e3f2 service swift] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:54.782135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-82677536-e787-47c4-9e3c-3aef8606e3f2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:54.852849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a534061-297b-48d2-99ff-3e788e4e0339 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:54.859357 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7169a5-86e6-4452-a25e-2cd938bba76e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:54.860035 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7169a5-86e6-4452-a25e-2cd938bba76e service placement] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.860326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7169a5-86e6-4452-a25e-2cd938bba76e service placement] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.860604 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7169a5-86e6-4452-a25e-2cd938bba76e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.862516 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-3b7169a5-86e6-4452-a25e-2cd938bba76e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:54.875880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7169a5-86e6-4452-a25e-2cd938bba76e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:54.877572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7169a5-86e6-4452-a25e-2cd938bba76e service placement] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:54.886315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82677536-e787-47c4-9e3c-3aef8606e3f2 service swift] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:54.893348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0d971-84b4-45c4-9470-5e258a6b3a5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:54.894119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0d971-84b4-45c4-9470-5e258a6b3a5e service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.894483 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0d971-84b4-45c4-9470-5e258a6b3a5e service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.894834 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0d971-84b4-45c4-9470-5e258a6b3a5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.901350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1161/2292] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:54.905520 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0d971-84b4-45c4-9470-5e258a6b3a5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:54.907304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0d971-84b4-45c4-9470-5e258a6b3a5e service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:54.920417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1132/2293] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:40:57.063359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb440e-94f4-4ce8-a0dc-993fdab1e0a5 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:57.063919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb440e-94f4-4ce8-a0dc-993fdab1e0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:57.064017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb440e-94f4-4ce8-a0dc-993fdab1e0a5 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:57.065824 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1162/2294] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jun 7 08:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:40:57.071968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25225d30-eafb-49bd-8f7e-46df7372ffa2 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:57.072454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25225d30-eafb-49bd-8f7e-46df7372ffa2 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:57.072570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25225d30-eafb-49bd-8f7e-46df7372ffa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:57.132956 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-25225d30-eafb-49bd-8f7e-46df7372ffa2 None None] MFA Rules not processed for user `3f2a2bf913d1412f996acf4401ba7673`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:57.134208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-25225d30-eafb-49bd-8f7e-46df7372ffa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:57.217791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1133/2295] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jun 7 08:40:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:57.230382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb440e-94f4-4ce8-a0dc-993fdab1e0a5 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:57.241296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb440e-94f4-4ce8-a0dc-993fdab1e0a5 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:57.244838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcfadd-0698-4b42-a299-54183ae5aa00 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e4531ac696445ba5bfa9870984befd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:57.245516 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcfadd-0698-4b42-a299-54183ae5aa00 service glance] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:57.245804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcfadd-0698-4b42-a299-54183ae5aa00 service glance] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:57.246085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcfadd-0698-4b42-a299-54183ae5aa00 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:57.256465 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcfadd-0698-4b42-a299-54183ae5aa00 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:57.257932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcfadd-0698-4b42-a299-54183ae5aa00 service glance] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:57.269214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1163/2296] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:57.335766 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9f199-a30c-478b-b094-f3cd9e662691 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:57.336062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9f199-a30c-478b-b094-f3cd9e662691 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:57.336352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9f199-a30c-478b-b094-f3cd9e662691 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:57.337762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1134/2297] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jun 7 08:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:40:57.343231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adba10-757f-4011-8831-3b6e17592d20 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:57.343523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adba10-757f-4011-8831-3b6e17592d20 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:57.343838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adba10-757f-4011-8831-3b6e17592d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:57.362267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e8adba10-757f-4011-8831-3b6e17592d20 None None] MFA Rules not processed for user `7ddf89deb28749eda2d26ad66bc282f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:57.363574 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e8adba10-757f-4011-8831-3b6e17592d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:57.416370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1164/2298] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jun 7 08:40:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:57.428064 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9f199-a30c-478b-b094-f3cd9e662691 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:57.429913 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-28b9f199-a30c-478b-b094-f3cd9e662691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:57.443049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9f199-a30c-478b-b094-f3cd9e662691 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:57.446602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fc236-a22c-4519-9c30-18bb09377e0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:57.447235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fc236-a22c-4519-9c30-18bb09377e0e service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:57.447522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fc236-a22c-4519-9c30-18bb09377e0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:57.447802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fc236-a22c-4519-9c30-18bb09377e0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:57.455521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fc236-a22c-4519-9c30-18bb09377e0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:57.456940 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fc236-a22c-4519-9c30-18bb09377e0e service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:57.464907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1135/2299] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:58.563819 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd5725-c89d-4406-a995-0f9bfea6b1e4 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:58.564329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd5725-c89d-4406-a995-0f9bfea6b1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:58.564444 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd5725-c89d-4406-a995-0f9bfea6b1e4 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:58.565804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1165/2300] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jun 7 08:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:40:58.570907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaefde-bf21-4c4a-bba3-f08034610271 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:58.571359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaefde-bf21-4c4a-bba3-f08034610271 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:58.571498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaefde-bf21-4c4a-bba3-f08034610271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:58.590870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c3eaefde-bf21-4c4a-bba3-f08034610271 None None] MFA Rules not processed for user `3f2a2bf913d1412f996acf4401ba7673`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:58.592211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c3eaefde-bf21-4c4a-bba3-f08034610271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:58.601945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1136/2301] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jun 7 08:40:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:58.616068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd5725-c89d-4406-a995-0f9bfea6b1e4 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:58.627043 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd5725-c89d-4406-a995-0f9bfea6b1e4 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:58.630603 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00daafc-2647-48d9-9ff0-77845cc98e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:58.631326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00daafc-2647-48d9-9ff0-77845cc98e2b service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:58.631603 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00daafc-2647-48d9-9ff0-77845cc98e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:58.631880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00daafc-2647-48d9-9ff0-77845cc98e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:58.643230 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00daafc-2647-48d9-9ff0-77845cc98e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:58.644667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00daafc-2647-48d9-9ff0-77845cc98e2b service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:58.656783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1166/2302] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:41:05.266071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d61d4-f903-409e-b95b-6715f4990e35 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:05.266613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d61d4-f903-409e-b95b-6715f4990e35 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:05.266725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d61d4-f903-409e-b95b-6715f4990e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:05.284600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-057d61d4-f903-409e-b95b-6715f4990e35 None None] MFA Rules not processed for user `3e6fe6fa3fc14cca97064b49403eb68b`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:05.285827 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-057d61d4-f903-409e-b95b-6715f4990e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:05.311464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:05.343552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1137/2303] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:41:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:08.650859 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=76 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:10.071694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00daafc-2647-48d9-9ff0-77845cc98e2b service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:10.073400 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e00daafc-2647-48d9-9ff0-77845cc98e2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:10.086068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00daafc-2647-48d9-9ff0-77845cc98e2b service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:10.090483 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e7905-01ea-456f-9874-b19623373426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:10.091208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e7905-01ea-456f-9874-b19623373426 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:10.091572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e7905-01ea-456f-9874-b19623373426 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:10.091917 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e7905-01ea-456f-9874-b19623373426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:10.093805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9d1e7905-01ea-456f-9874-b19623373426 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:10.108795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e7905-01ea-456f-9874-b19623373426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:10.110276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e7905-01ea-456f-9874-b19623373426 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:10.122139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1167/2304] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:41:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:41:15.335580 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=12 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:20.116808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:25.479932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d61d4-f903-409e-b95b-6715f4990e35 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:25.481513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-057d61d4-f903-409e-b95b-6715f4990e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:25.498682 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d61d4-f903-409e-b95b-6715f4990e35 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:25.504311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf206d4-6287-4c9f-8519-e1c7b49e5c23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:25.504975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf206d4-6287-4c9f-8519-e1c7b49e5c23 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:25.505290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf206d4-6287-4c9f-8519-e1c7b49e5c23 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:25.505570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf206d4-6287-4c9f-8519-e1c7b49e5c23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:25.507381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cbf206d4-6287-4c9f-8519-e1c7b49e5c23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:25.572551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf206d4-6287-4c9f-8519-e1c7b49e5c23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:25.574214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf206d4-6287-4c9f-8519-e1c7b49e5c23 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:25.585259 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1138/2305] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:41:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:41:33.882501 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e7905-01ea-456f-9874-b19623373426 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:33.883937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9d1e7905-01ea-456f-9874-b19623373426 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:33.941866 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e7905-01ea-456f-9874-b19623373426 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:33.945450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198636e-b820-414c-bf6e-e0a43c217efb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:33.946084 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198636e-b820-414c-bf6e-e0a43c217efb service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:33.946366 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198636e-b820-414c-bf6e-e0a43c217efb service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:33.946638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198636e-b820-414c-bf6e-e0a43c217efb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:33.948380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-7198636e-b820-414c-bf6e-e0a43c217efb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:34.007787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198636e-b820-414c-bf6e-e0a43c217efb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:34.009265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198636e-b820-414c-bf6e-e0a43c217efb service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:34.019579 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1168/2306] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:41:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:41:35.580664 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=28 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:41.490969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf206d4-6287-4c9f-8519-e1c7b49e5c23 service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:41.500732 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf206d4-6287-4c9f-8519-e1c7b49e5c23 service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:41.504326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ab0de-24cc-4ba1-b77b-acd736ad3598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:41.504967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ab0de-24cc-4ba1-b77b-acd736ad3598 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:41.505287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ab0de-24cc-4ba1-b77b-acd736ad3598 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:41.505569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ab0de-24cc-4ba1-b77b-acd736ad3598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:41.507317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5a6ab0de-24cc-4ba1-b77b-acd736ad3598 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:41.519985 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ab0de-24cc-4ba1-b77b-acd736ad3598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:41.521428 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ab0de-24cc-4ba1-b77b-acd736ad3598 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:41.530905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1139/2307] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:41:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:41:43.158163 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198636e-b820-414c-bf6e-e0a43c217efb service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:43.167458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198636e-b820-414c-bf6e-e0a43c217efb service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:43.171048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b261ccf-3e49-4404-a83b-d7985a08f71e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:43.171677 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b261ccf-3e49-4404-a83b-d7985a08f71e service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:43.171957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b261ccf-3e49-4404-a83b-d7985a08f71e service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:43.172237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b261ccf-3e49-4404-a83b-d7985a08f71e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:43.173977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8b261ccf-3e49-4404-a83b-d7985a08f71e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:43.187506 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b261ccf-3e49-4404-a83b-d7985a08f71e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:43.188922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b261ccf-3e49-4404-a83b-d7985a08f71e service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:43.200637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1169/2308] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:41:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:41:51.526365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:53.194912 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=54 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:55.408615 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ab0de-24cc-4ba1-b77b-acd736ad3598 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:55.418483 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ab0de-24cc-4ba1-b77b-acd736ad3598 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:55.422116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5658cb8-6b20-4d20-9b84-8017438fd3b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:55.422744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5658cb8-6b20-4d20-9b84-8017438fd3b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:55.423023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5658cb8-6b20-4d20-9b84-8017438fd3b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:55.423296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5658cb8-6b20-4d20-9b84-8017438fd3b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:55.425060 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a5658cb8-6b20-4d20-9b84-8017438fd3b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:55.435363 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5658cb8-6b20-4d20-9b84-8017438fd3b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:55.436771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5658cb8-6b20-4d20-9b84-8017438fd3b8 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:55.444770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1140/2309] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:41:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:42:05.441276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:42:38.436419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b261ccf-3e49-4404-a83b-d7985a08f71e service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:42:38.447832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b261ccf-3e49-4404-a83b-d7985a08f71e service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:42:38.459375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd7936-faf1-4268-91d3-8cb9735a7256 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:42:38.460283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd7936-faf1-4268-91d3-8cb9735a7256 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:42:38.460567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd7936-faf1-4268-91d3-8cb9735a7256 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:42:38.460866 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd7936-faf1-4268-91d3-8cb9735a7256 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:42:38.462772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8abd7936-faf1-4268-91d3-8cb9735a7256 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:42:38.525329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd7936-faf1-4268-91d3-8cb9735a7256 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:42:38.526914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd7936-faf1-4268-91d3-8cb9735a7256 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:42:38.536487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1170/2310] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:42:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:42:48.532492 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=28 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:09.159492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5658cb8-6b20-4d20-9b84-8017438fd3b8 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:09.161870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a5658cb8-6b20-4d20-9b84-8017438fd3b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:09.173431 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5658cb8-6b20-4d20-9b84-8017438fd3b8 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:09.177756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0129c-bb45-45a6-88dd-9ad8bf09f326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:09.178434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0129c-bb45-45a6-88dd-9ad8bf09f326 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:09.178718 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0129c-bb45-45a6-88dd-9ad8bf09f326 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:09.178996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0129c-bb45-45a6-88dd-9ad8bf09f326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:09.180875 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-62e0129c-bb45-45a6-88dd-9ad8bf09f326 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:09.245786 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0129c-bb45-45a6-88dd-9ad8bf09f326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:09.247318 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0129c-bb45-45a6-88dd-9ad8bf09f326 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:09.257212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1141/2311] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:43:19.253240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=28 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:36.697521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd7936-faf1-4268-91d3-8cb9735a7256 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:36.707274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd7936-faf1-4268-91d3-8cb9735a7256 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:36.711160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68ee52-796f-4d73-b7e7-fd483d01703a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:36.711794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68ee52-796f-4d73-b7e7-fd483d01703a service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:36.712077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68ee52-796f-4d73-b7e7-fd483d01703a service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:36.712349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68ee52-796f-4d73-b7e7-fd483d01703a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:36.714144 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-df68ee52-796f-4d73-b7e7-fd483d01703a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:36.773135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68ee52-796f-4d73-b7e7-fd483d01703a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:36.774574 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68ee52-796f-4d73-b7e7-fd483d01703a service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:36.783979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1171/2312] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:36 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:43:46.779394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=28 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:02.978714 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0129c-bb45-45a6-88dd-9ad8bf09f326 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:02.987221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0129c-bb45-45a6-88dd-9ad8bf09f326 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:02.990812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fab80-3ca5-4524-8346-f8e201c4f22d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:02.991456 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fab80-3ca5-4524-8346-f8e201c4f22d service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:02.991745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fab80-3ca5-4524-8346-f8e201c4f22d service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:02.992025 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fab80-3ca5-4524-8346-f8e201c4f22d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:03.001963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fab80-3ca5-4524-8346-f8e201c4f22d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:03.003485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fab80-3ca5-4524-8346-f8e201c4f22d service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:03.013149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1142/2313] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:02 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:13.009109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:42.241912 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68ee52-796f-4d73-b7e7-fd483d01703a service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:42.243339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-df68ee52-796f-4d73-b7e7-fd483d01703a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:42.318296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68ee52-796f-4d73-b7e7-fd483d01703a service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:42.326593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:42.327516 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:42.327804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:42.328081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] PATH_INFO: `/v3/users/8ab6821a01c8465ab37e3304f1e26f6f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:42.329336 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab6821a01c8465ab37e3304f1e26f6f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:42.331299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:42.385457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab6821a01c8465ab37e3304f1e26f6f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:42.386171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab6821a01c8465ab37e3304f1e26f6f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:42.394613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab6821a01c8465ab37e3304f1e26f6f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:42.413727 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab6821a01c8465ab37e3304f1e26f6f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:42.414690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab6821a01c8465ab37e3304f1e26f6f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:42.439981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1172/2314] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:45:42 2022] DELETE /identity/v3/users/8ab6821a01c8465ab37e3304f1e26f6f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:45:42.440085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42c86a-4beb-4815-9dc7-e8f83f0121b3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:42.440085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42c86a-4beb-4815-9dc7-e8f83f0121b3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:42.440085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42c86a-4beb-4815-9dc7-e8f83f0121b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:42.455723 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5a42c86a-4beb-4815-9dc7-e8f83f0121b3 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:42.457029 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5a42c86a-4beb-4815-9dc7-e8f83f0121b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:42.518154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1143/2315] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:45:42.527896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:42.530056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:42.586861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51660bbb-ba09-4caf-819f-48d65c9c8e40 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:42.590547 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c9817-b95d-480d-a5ec-9950120afc33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:42.591749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c9817-b95d-480d-a5ec-9950120afc33 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:42.592035 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c9817-b95d-480d-a5ec-9950120afc33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:42.592312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c9817-b95d-480d-a5ec-9950120afc33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:42.601620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c9817-b95d-480d-a5ec-9950120afc33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:42.603002 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c9817-b95d-480d-a5ec-9950120afc33 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:42.612902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1173/2316] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:45:42.928445 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42c86a-4beb-4815-9dc7-e8f83f0121b3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:42.929947 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5a42c86a-4beb-4815-9dc7-e8f83f0121b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:42.947614 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42c86a-4beb-4815-9dc7-e8f83f0121b3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:42.955492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:42.956501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:42.956650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:42.956930 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] PATH_INFO: `/v3/projects/e891f44f3e7f4abdbaa4cf938f315a15` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:42.958160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e891f44f3e7f4abdbaa4cf938f315a15)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:42.960071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:43.010789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e891f44f3e7f4abdbaa4cf938f315a15', 'request_id': 'req-1fe2198b-745d-4215-8a1f-3392052530f3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:43.017182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e891f44f3e7f4abdbaa4cf938f315a15', 'request_id': 'req-1fe2198b-745d-4215-8a1f-3392052530f3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:43.025173 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e891f44f3e7f4abdbaa4cf938f315a15', 'request_id': 'req-1fe2198b-745d-4215-8a1f-3392052530f3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:43.036873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] The token cache is being invalidate because project e891f44f3e7f4abdbaa4cf938f315a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:45:43.037198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:43.037847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:43.039389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1144/2317] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:45:42 2022] DELETE /identity/v3/projects/e891f44f3e7f4abdbaa4cf938f315a15 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:45:52.608482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:53.033745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:10.936505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c9817-b95d-480d-a5ec-9950120afc33 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:10.948617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c9817-b95d-480d-a5ec-9950120afc33 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:11.001871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c5c88-1f03-4b9c-b621-5a0684f02ce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:11.002536 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c5c88-1f03-4b9c-b621-5a0684f02ce2 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:11.002908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c5c88-1f03-4b9c-b621-5a0684f02ce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:11.003191 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c5c88-1f03-4b9c-b621-5a0684f02ce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:11.005040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-679c5c88-1f03-4b9c-b621-5a0684f02ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:11.095660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c5c88-1f03-4b9c-b621-5a0684f02ce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:11.097155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c5c88-1f03-4b9c-b621-5a0684f02ce2 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:11.109159 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1174/2318] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:46:14.571444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:14.572921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:14.644518 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2198b-745d-4215-8a1f-3392052530f3 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:14.650935 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2adc1-5d83-4563-93a1-0d0acdc70cf4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:14.651662 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2adc1-5d83-4563-93a1-0d0acdc70cf4 service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:14.652630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2adc1-5d83-4563-93a1-0d0acdc70cf4 service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:14.652630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2adc1-5d83-4563-93a1-0d0acdc70cf4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:14.656885 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bfe2adc1-5d83-4563-93a1-0d0acdc70cf4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:14.670560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2adc1-5d83-4563-93a1-0d0acdc70cf4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:14.672234 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2adc1-5d83-4563-93a1-0d0acdc70cf4 service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:14.683616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1145/2319] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:46:21.103772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=54 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:24.679010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=32 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:25.482413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c5c88-1f03-4b9c-b621-5a0684f02ce2 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:25.483938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-679c5c88-1f03-4b9c-b621-5a0684f02ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:25.498883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c5c88-1f03-4b9c-b621-5a0684f02ce2 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:25.505034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67dcf0-466e-4b01-8aaa-9b65dcdbde06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:25.505719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67dcf0-466e-4b01-8aaa-9b65dcdbde06 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:25.506007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67dcf0-466e-4b01-8aaa-9b65dcdbde06 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:25.506288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67dcf0-466e-4b01-8aaa-9b65dcdbde06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:25.508089 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4b67dcf0-466e-4b01-8aaa-9b65dcdbde06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:25.600983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67dcf0-466e-4b01-8aaa-9b65dcdbde06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:25.602611 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67dcf0-466e-4b01-8aaa-9b65dcdbde06 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:25.622503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1175/2320] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:46:33.837496 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2adc1-5d83-4563-93a1-0d0acdc70cf4 service placement] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:33.850221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2adc1-5d83-4563-93a1-0d0acdc70cf4 service placement] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:33.855839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979d68d-f92d-414a-aae2-e5146fa36b59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:33.856482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979d68d-f92d-414a-aae2-e5146fa36b59 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:33.856798 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979d68d-f92d-414a-aae2-e5146fa36b59 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:33.857103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979d68d-f92d-414a-aae2-e5146fa36b59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:33.858899 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5979d68d-f92d-414a-aae2-e5146fa36b59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:33.948051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979d68d-f92d-414a-aae2-e5146fa36b59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:33.949686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979d68d-f92d-414a-aae2-e5146fa36b59 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:33.970347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1146/2321] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:46:35.619832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=41 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:42.392719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67dcf0-466e-4b01-8aaa-9b65dcdbde06 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:42.394265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4b67dcf0-466e-4b01-8aaa-9b65dcdbde06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:42.409015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67dcf0-466e-4b01-8aaa-9b65dcdbde06 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:42.415372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed23a07-adba-4ac2-a7d0-ff23a0325764 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:42.416019 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed23a07-adba-4ac2-a7d0-ff23a0325764 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:42.416301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed23a07-adba-4ac2-a7d0-ff23a0325764 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:42.416575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed23a07-adba-4ac2-a7d0-ff23a0325764 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:42.418324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-bed23a07-adba-4ac2-a7d0-ff23a0325764 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:42.509732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed23a07-adba-4ac2-a7d0-ff23a0325764 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:42.511367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed23a07-adba-4ac2-a7d0-ff23a0325764 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:42.534283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1176/2322] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:46:43.966757 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=41 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:52.531461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=41 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:58.082861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979d68d-f92d-414a-aae2-e5146fa36b59 service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:58.085279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5979d68d-f92d-414a-aae2-e5146fa36b59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:58.153650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979d68d-f92d-414a-aae2-e5146fa36b59 service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:58.157297 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167ac21-8526-4274-a6f5-50096929f2f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:58.157957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167ac21-8526-4274-a6f5-50096929f2f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:58.158376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167ac21-8526-4274-a6f5-50096929f2f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:58.158557 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167ac21-8526-4274-a6f5-50096929f2f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:58.160473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0167ac21-8526-4274-a6f5-50096929f2f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:58.175352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167ac21-8526-4274-a6f5-50096929f2f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:58.176795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167ac21-8526-4274-a6f5-50096929f2f2 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:58.190598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1147/2323] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:47:08.183513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=26 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:47:18.304658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed23a07-adba-4ac2-a7d0-ff23a0325764 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:18.306015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-bed23a07-adba-4ac2-a7d0-ff23a0325764 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:18.317623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed23a07-adba-4ac2-a7d0-ff23a0325764 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:18.321177 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1bc32-eaed-46c9-a7b0-64e6b727d7e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:18.321812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1bc32-eaed-46c9-a7b0-64e6b727d7e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:18.322121 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1bc32-eaed-46c9-a7b0-64e6b727d7e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:18.322370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1bc32-eaed-46c9-a7b0-64e6b727d7e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:18.324171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-92c1bc32-eaed-46c9-a7b0-64e6b727d7e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:18.334190 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1bc32-eaed-46c9-a7b0-64e6b727d7e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:18.335605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1bc32-eaed-46c9-a7b0-64e6b727d7e2 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:18.343770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1177/2324] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:47:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:47:28.340210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:47:38.465870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167ac21-8526-4274-a6f5-50096929f2f2 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:38.477946 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167ac21-8526-4274-a6f5-50096929f2f2 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:38.483602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbd723-bed0-4210-b156-2fa9c37aaee6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:38.484243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbd723-bed0-4210-b156-2fa9c37aaee6 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:38.484580 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbd723-bed0-4210-b156-2fa9c37aaee6 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:38.484800 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbd723-bed0-4210-b156-2fa9c37aaee6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:38.486720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dbcbd723-bed0-4210-b156-2fa9c37aaee6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:38.567915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbd723-bed0-4210-b156-2fa9c37aaee6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:38.569533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbd723-bed0-4210-b156-2fa9c37aaee6 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:38.588397 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1148/2325] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:47:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:47:48.585502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=41 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:48:09.675623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1bc32-eaed-46c9-a7b0-64e6b727d7e2 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:48:09.695763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1bc32-eaed-46c9-a7b0-64e6b727d7e2 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:48:09.699421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b383899-36ae-4272-bad7-f8430580bf97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:48:09.700062 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b383899-36ae-4272-bad7-f8430580bf97 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:48:09.700350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b383899-36ae-4272-bad7-f8430580bf97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:48:09.700631 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b383899-36ae-4272-bad7-f8430580bf97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:48:09.702448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0b383899-36ae-4272-bad7-f8430580bf97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:48:09.807952 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b383899-36ae-4272-bad7-f8430580bf97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:48:09.809438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b383899-36ae-4272-bad7-f8430580bf97 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:48:09.827935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1178/2326] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:48:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:48:19.825257 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=56 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:49:03.680007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbd723-bed0-4210-b156-2fa9c37aaee6 service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:49:03.681528 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dbcbd723-bed0-4210-b156-2fa9c37aaee6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:49:03.747240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbd723-bed0-4210-b156-2fa9c37aaee6 service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:49:03.750948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a59fac-1d1c-46cd-b47f-b5e0a1f0edd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:49:03.751604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a59fac-1d1c-46cd-b47f-b5e0a1f0edd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:49:03.751892 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a59fac-1d1c-46cd-b47f-b5e0a1f0edd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:49:03.752174 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a59fac-1d1c-46cd-b47f-b5e0a1f0edd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:49:03.753968 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-30a59fac-1d1c-46cd-b47f-b5e0a1f0edd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:49:03.833207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a59fac-1d1c-46cd-b47f-b5e0a1f0edd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:49:03.834634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a59fac-1d1c-46cd-b47f-b5e0a1f0edd5 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:49:03.853878 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1149/2327] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:49:03 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:49:13.852915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=63 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:20.578034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b383899-36ae-4272-bad7-f8430580bf97 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:20.579443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0b383899-36ae-4272-bad7-f8430580bf97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:20.622831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b383899-36ae-4272-bad7-f8430580bf97 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:20.628085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd4028-ce83-40c3-b9a5-64e55c4df8ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:20.628731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd4028-ce83-40c3-b9a5-64e55c4df8ff service placement] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:20.629019 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd4028-ce83-40c3-b9a5-64e55c4df8ff service placement] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:20.629331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd4028-ce83-40c3-b9a5-64e55c4df8ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:20.638225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd4028-ce83-40c3-b9a5-64e55c4df8ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:20.639845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd4028-ce83-40c3-b9a5-64e55c4df8ff service placement] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:20.650275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1179/2328] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:51:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:51:26.472378 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a59fac-1d1c-46cd-b47f-b5e0a1f0edd5 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:26.481180 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a59fac-1d1c-46cd-b47f-b5e0a1f0edd5 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:26.484690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d734633-b295-4caa-83e8-be1ad5737bad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:26.485363 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d734633-b295-4caa-83e8-be1ad5737bad service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:26.485648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d734633-b295-4caa-83e8-be1ad5737bad service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:26.485925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d734633-b295-4caa-83e8-be1ad5737bad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:26.494384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d734633-b295-4caa-83e8-be1ad5737bad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:26.495799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d734633-b295-4caa-83e8-be1ad5737bad service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:26.504786 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1150/2329] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:51:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:51:30.645488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=18 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:33.311827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd4028-ce83-40c3-b9a5-64e55c4df8ff service placement] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:33.341523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd4028-ce83-40c3-b9a5-64e55c4df8ff service placement] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:33.347045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c0051-d621-4bf4-838d-d201d74ef0e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:33.347688 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c0051-d621-4bf4-838d-d201d74ef0e4 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:33.347974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c0051-d621-4bf4-838d-d201d74ef0e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:33.348253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c0051-d621-4bf4-838d-d201d74ef0e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:33.357259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c0051-d621-4bf4-838d-d201d74ef0e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:33.358907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c0051-d621-4bf4-838d-d201d74ef0e4 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:33.368768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1180/2330] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:51:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:51:36.500637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:42.296231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d734633-b295-4caa-83e8-be1ad5737bad service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:42.297773 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4d734633-b295-4caa-83e8-be1ad5737bad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:42.332434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d734633-b295-4caa-83e8-be1ad5737bad service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:42.335996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f3e19-92c3-47ff-9d68-bee546f73c2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:42.336648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f3e19-92c3-47ff-9d68-bee546f73c2e service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:42.336930 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f3e19-92c3-47ff-9d68-bee546f73c2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:42.337253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f3e19-92c3-47ff-9d68-bee546f73c2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:42.348784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f3e19-92c3-47ff-9d68-bee546f73c2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:42.350372 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f3e19-92c3-47ff-9d68-bee546f73c2e service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:42.362343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1151/2331] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:51:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:51:42.429746 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c0051-d621-4bf4-838d-d201d74ef0e4 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:42.438996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c0051-d621-4bf4-838d-d201d74ef0e4 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:42.442792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276775f-ce48-4ca9-9e09-cb6db70cbd10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:42.443431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276775f-ce48-4ca9-9e09-cb6db70cbd10 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:42.443714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276775f-ce48-4ca9-9e09-cb6db70cbd10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:42.443992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276775f-ce48-4ca9-9e09-cb6db70cbd10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:42.453061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276775f-ce48-4ca9-9e09-cb6db70cbd10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:42.454533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276775f-ce48-4ca9-9e09-cb6db70cbd10 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:42.464770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1181/2332] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:51:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:51:52.357323 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=15 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:52.459531 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=19 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:52:22.312535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f3e19-92c3-47ff-9d68-bee546f73c2e service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:52:22.321307 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f3e19-92c3-47ff-9d68-bee546f73c2e service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:52:22.325028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c214a8-b815-4f10-a1cd-3fdd655978d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:52:22.326202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c214a8-b815-4f10-a1cd-3fdd655978d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:52:22.326495 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c214a8-b815-4f10-a1cd-3fdd655978d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:52:22.326773 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c214a8-b815-4f10-a1cd-3fdd655978d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:52:22.335089 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c214a8-b815-4f10-a1cd-3fdd655978d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:52:22.336503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c214a8-b815-4f10-a1cd-3fdd655978d3 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:52:22.345523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1152/2333] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:52:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:52:32.341321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:52:37.845570 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276775f-ce48-4ca9-9e09-cb6db70cbd10 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:52:37.859482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276775f-ce48-4ca9-9e09-cb6db70cbd10 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:52:37.865560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfee7e-3d7a-49ca-999c-0b25e00eebd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:52:37.866244 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfee7e-3d7a-49ca-999c-0b25e00eebd9 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:52:37.866533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfee7e-3d7a-49ca-999c-0b25e00eebd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:52:37.866811 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfee7e-3d7a-49ca-999c-0b25e00eebd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:52:37.875822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfee7e-3d7a-49ca-999c-0b25e00eebd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:52:37.877487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfee7e-3d7a-49ca-999c-0b25e00eebd9 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:52:37.888966 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1182/2334] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:52:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:52:47.884247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:52:59.556804 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c214a8-b815-4f10-a1cd-3fdd655978d3 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:52:59.570386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c214a8-b815-4f10-a1cd-3fdd655978d3 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:52:59.574396 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b3e06-1c2b-4910-ab1c-ce166f0f4305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:52:59.575975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b3e06-1c2b-4910-ab1c-ce166f0f4305 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:52:59.575975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b3e06-1c2b-4910-ab1c-ce166f0f4305 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:52:59.576206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b3e06-1c2b-4910-ab1c-ce166f0f4305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:52:59.587595 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b3e06-1c2b-4910-ab1c-ce166f0f4305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:52:59.589396 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b3e06-1c2b-4910-ab1c-ce166f0f4305 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:52:59.601097 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1153/2335] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:52:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:53:09.596014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:53:10.070580 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfee7e-3d7a-49ca-999c-0b25e00eebd9 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:53:10.082103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfee7e-3d7a-49ca-999c-0b25e00eebd9 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:53:10.087564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fcd33-c955-4991-838c-cf12d3a9b7a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:53:10.088198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fcd33-c955-4991-838c-cf12d3a9b7a4 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:53:10.088487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fcd33-c955-4991-838c-cf12d3a9b7a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:53:10.089023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fcd33-c955-4991-838c-cf12d3a9b7a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:53:10.097640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fcd33-c955-4991-838c-cf12d3a9b7a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:53:10.099261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fcd33-c955-4991-838c-cf12d3a9b7a4 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:53:10.108242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1183/2336] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:53:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:53:20.104335 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:54:03.489248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b3e06-1c2b-4910-ab1c-ce166f0f4305 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:54:03.498533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b3e06-1c2b-4910-ab1c-ce166f0f4305 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:54:03.503181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17fe6b-93be-48bf-8508-2fdf6517c547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:54:03.503821 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17fe6b-93be-48bf-8508-2fdf6517c547 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:54:03.504105 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17fe6b-93be-48bf-8508-2fdf6517c547 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:54:03.504377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17fe6b-93be-48bf-8508-2fdf6517c547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:54:03.513289 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17fe6b-93be-48bf-8508-2fdf6517c547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:54:03.515536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17fe6b-93be-48bf-8508-2fdf6517c547 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:54:03.525435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1154/2337] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:54:03 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:54:13.520799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:56:25.199260 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fcd33-c955-4991-838c-cf12d3a9b7a4 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:56:25.271279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fcd33-c955-4991-838c-cf12d3a9b7a4 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:56:25.276873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f917a1-06a1-440e-894e-f89ec3c4e235 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:56:25.277565 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f917a1-06a1-440e-894e-f89ec3c4e235 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:56:25.277853 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f917a1-06a1-440e-894e-f89ec3c4e235 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:56:25.278135 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f917a1-06a1-440e-894e-f89ec3c4e235 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:56:25.286282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f917a1-06a1-440e-894e-f89ec3c4e235 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:56:25.287856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f917a1-06a1-440e-894e-f89ec3c4e235 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:56:25.297412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1184/2338] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:56:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:56:26.597439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17fe6b-93be-48bf-8508-2fdf6517c547 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:56:26.600943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ef17fe6b-93be-48bf-8508-2fdf6517c547 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:56:26.683758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17fe6b-93be-48bf-8508-2fdf6517c547 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:56:26.689252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c472b5f-082c-40cf-a70f-e7a84a120251 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:56:26.689910 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c472b5f-082c-40cf-a70f-e7a84a120251 service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:56:26.690197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c472b5f-082c-40cf-a70f-e7a84a120251 service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:56:26.690475 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c472b5f-082c-40cf-a70f-e7a84a120251 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:56:26.692548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2c472b5f-082c-40cf-a70f-e7a84a120251 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:56:26.706213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c472b5f-082c-40cf-a70f-e7a84a120251 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:56:26.707898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c472b5f-082c-40cf-a70f-e7a84a120251 service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:56:26.719436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1155/2339] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:56:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:56:33.246675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f917a1-06a1-440e-894e-f89ec3c4e235 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:56:33.256980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f917a1-06a1-440e-894e-f89ec3c4e235 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:56:33.260745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa729d-4100-4edf-bf3f-e1ddbd41217a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:56:33.261413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa729d-4100-4edf-bf3f-e1ddbd41217a service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:56:33.261701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa729d-4100-4edf-bf3f-e1ddbd41217a service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:56:33.261976 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa729d-4100-4edf-bf3f-e1ddbd41217a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:56:33.271192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa729d-4100-4edf-bf3f-e1ddbd41217a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:56:33.272619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa729d-4100-4edf-bf3f-e1ddbd41217a service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:56:33.281695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1185/2340] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:56:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:56:36.714085 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=32 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:56:41.810709 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c472b5f-082c-40cf-a70f-e7a84a120251 service placement] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:56:41.813402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2c472b5f-082c-40cf-a70f-e7a84a120251 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:56:41.841691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c472b5f-082c-40cf-a70f-e7a84a120251 service placement] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:56:41.847515 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e15c5a-77f7-4820-915e-598084704e79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:56:41.848163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e15c5a-77f7-4820-915e-598084704e79 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:56:41.848443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e15c5a-77f7-4820-915e-598084704e79 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:56:41.848712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e15c5a-77f7-4820-915e-598084704e79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:56:41.857334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e15c5a-77f7-4820-915e-598084704e79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:56:41.860053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e15c5a-77f7-4820-915e-598084704e79 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:56:41.869331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1156/2341] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:56:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:56:41.920647 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa729d-4100-4edf-bf3f-e1ddbd41217a service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:56:41.977036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa729d-4100-4edf-bf3f-e1ddbd41217a service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:56:41.980686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1173c76-ef0f-4005-8d4a-3e25a5f7326a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:56:41.981361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1173c76-ef0f-4005-8d4a-3e25a5f7326a service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:56:41.981653 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1173c76-ef0f-4005-8d4a-3e25a5f7326a service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:56:41.981940 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1173c76-ef0f-4005-8d4a-3e25a5f7326a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:56:41.984539 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f1173c76-ef0f-4005-8d4a-3e25a5f7326a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:56:41.998096 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1173c76-ef0f-4005-8d4a-3e25a5f7326a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:56:42.000245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1173c76-ef0f-4005-8d4a-3e25a5f7326a service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:56:42.011208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1186/2342] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:56:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:56:51.864890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=11 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:56:52.006406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=66 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:57:28.305482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e15c5a-77f7-4820-915e-598084704e79 service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:28.306993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-16e15c5a-77f7-4820-915e-598084704e79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:28.319560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e15c5a-77f7-4820-915e-598084704e79 service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:28.323154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85994af6-d15a-4396-aaf4-a76445748519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:28.323789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85994af6-d15a-4396-aaf4-a76445748519 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:28.324076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85994af6-d15a-4396-aaf4-a76445748519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:28.324356 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85994af6-d15a-4396-aaf4-a76445748519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:28.326204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-85994af6-d15a-4396-aaf4-a76445748519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:28.335704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85994af6-d15a-4396-aaf4-a76445748519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:28.337122 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85994af6-d15a-4396-aaf4-a76445748519 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:28.345423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1157/2343] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:57:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:57:38.341767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:57:38.531338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1173c76-ef0f-4005-8d4a-3e25a5f7326a service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:38.532930 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f1173c76-ef0f-4005-8d4a-3e25a5f7326a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:38.590175 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1173c76-ef0f-4005-8d4a-3e25a5f7326a service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:38.595324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628b9f0-57e2-4357-9b41-4802e8f88211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:38.595988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628b9f0-57e2-4357-9b41-4802e8f88211 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:38.596399 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628b9f0-57e2-4357-9b41-4802e8f88211 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:38.596717 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628b9f0-57e2-4357-9b41-4802e8f88211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:38.598674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d628b9f0-57e2-4357-9b41-4802e8f88211 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:38.681301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628b9f0-57e2-4357-9b41-4802e8f88211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:38.682743 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628b9f0-57e2-4357-9b41-4802e8f88211 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:38.701016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1187/2344] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:57:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:57:43.047402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85994af6-d15a-4396-aaf4-a76445748519 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:43.048815 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-85994af6-d15a-4396-aaf4-a76445748519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:43.145740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85994af6-d15a-4396-aaf4-a76445748519 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:43.150293 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:43.151158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:43.151440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:43.151717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] PATH_INFO: `/v3/users/4a27782a48ec45f8939f5015af3dbaa9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:43.152903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a27782a48ec45f8939f5015af3dbaa9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:43.154853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:43.201160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a27782a48ec45f8939f5015af3dbaa9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:57:43.201821 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a27782a48ec45f8939f5015af3dbaa9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:57:43.209446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a27782a48ec45f8939f5015af3dbaa9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:57:43.210110 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a27782a48ec45f8939f5015af3dbaa9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:57:43.222393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a27782a48ec45f8939f5015af3dbaa9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:57:43.235136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1158/2345] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 08:57:43 2022] DELETE /identity/v3/users/4a27782a48ec45f8939f5015af3dbaa9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:57:43.240134 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e60186-71fb-4fe7-9459-965dffa9c637 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:43.240732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e60186-71fb-4fe7-9459-965dffa9c637 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:43.240964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e60186-71fb-4fe7-9459-965dffa9c637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:43.268350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d8e60186-71fb-4fe7-9459-965dffa9c637 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:57:43.269712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d8e60186-71fb-4fe7-9459-965dffa9c637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:43.328434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1188/2346] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:57:43.337950 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:43.339338 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:43.397615 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab43f9-f951-412a-9106-5a8704028939 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:43.401274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08807f0f-2627-49c3-8d37-2992d40a7121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:43.401920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08807f0f-2627-49c3-8d37-2992d40a7121 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:43.402207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08807f0f-2627-49c3-8d37-2992d40a7121 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:43.402487 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08807f0f-2627-49c3-8d37-2992d40a7121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:43.409118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:57:43.413981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08807f0f-2627-49c3-8d37-2992d40a7121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:43.415535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08807f0f-2627-49c3-8d37-2992d40a7121 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:43.426482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1159/2347] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:57:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:57:43.696468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e60186-71fb-4fe7-9459-965dffa9c637 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:43.697919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d8e60186-71fb-4fe7-9459-965dffa9c637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:43.710350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e60186-71fb-4fe7-9459-965dffa9c637 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:43.715229 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:43.716450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:43.716679 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:43.716956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25 admin admin] PATH_INFO: `/v3/projects/9a574283f4d74e389163486ccc060d81` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:43.718954 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a574283f4d74e389163486ccc060d81)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:43.720964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:43.738540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:57:43.770016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a574283f4d74e389163486ccc060d81', 'request_id': 'req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:57:43.775906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a574283f4d74e389163486ccc060d81', 'request_id': 'req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:57:43.784693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a574283f4d74e389163486ccc060d81', 'request_id': 'req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:57:43.795247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25 admin admin] The token cache is being invalidate because project 9a574283f4d74e389163486ccc060d81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:57:43.795533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:57:43.796691 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-79ea3a7e-6c74-4e7c-ba7c-b1ec6a954c25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:57:43.798272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1189/2348] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 08:57:43 2022] DELETE /identity/v3/projects/9a574283f4d74e389163486ccc060d81 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:57:43.804195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef78a-5882-4a93-930e-98002a25620c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:43.804491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef78a-5882-4a93-930e-98002a25620c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:43.804776 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef78a-5882-4a93-930e-98002a25620c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:43.851733 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-017ef78a-5882-4a93-930e-98002a25620c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:57:43.852943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-017ef78a-5882-4a93-930e-98002a25620c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:43.914622 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1160/2349] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:57:43.922664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd76c30-0344-403a-a2b7-446a66e60500 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:43.923284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd76c30-0344-403a-a2b7-446a66e60500 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:43.923874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd76c30-0344-403a-a2b7-446a66e60500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:43.943878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-8cd76c30-0344-403a-a2b7-446a66e60500 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:57:43.945334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8cd76c30-0344-403a-a2b7-446a66e60500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:43.954211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1190/2350] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:57:43.958338 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ef78a-5882-4a93-930e-98002a25620c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:43.969712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ef78a-5882-4a93-930e-98002a25620c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:43.977866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42897e0e-e61d-47a1-a81e-9af53d757316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:43.978550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42897e0e-e61d-47a1-a81e-9af53d757316 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:43.979530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42897e0e-e61d-47a1-a81e-9af53d757316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:43.979530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42897e0e-e61d-47a1-a81e-9af53d757316 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:43.979834 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42897e0e-e61d-47a1-a81e-9af53d757316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:57:43.980136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42897e0e-e61d-47a1-a81e-9af53d757316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:43.982141 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42897e0e-e61d-47a1-a81e-9af53d757316 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:43.995400 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1161/2351] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 08:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:57:44.000966 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd76c30-0344-403a-a2b7-446a66e60500 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:44.012245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd76c30-0344-403a-a2b7-446a66e60500 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:44.018565 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3498d-99cc-46bd-8882-98c21c626cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:44.019336 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3498d-99cc-46bd-8882-98c21c626cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:44.020267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3498d-99cc-46bd-8882-98c21c626cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:44.020267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3498d-99cc-46bd-8882-98c21c626cf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:44.020826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3498d-99cc-46bd-8882-98c21c626cf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:44.022953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3498d-99cc-46bd-8882-98c21c626cf3 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:44.085509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1191/2352] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 08:57:43 2022] POST /identity/v3/projects => generated 391 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:57:44.089811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42897e0e-e61d-47a1-a81e-9af53d757316 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:44.100825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42897e0e-e61d-47a1-a81e-9af53d757316 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:44.159835 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5755c2-49ad-40ca-8b0b-8974a633d9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:44.160537 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5755c2-49ad-40ca-8b0b-8974a633d9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:44.160847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5755c2-49ad-40ca-8b0b-8974a633d9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:44.161153 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5755c2-49ad-40ca-8b0b-8974a633d9d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:44.161839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5755c2-49ad-40ca-8b0b-8974a633d9d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:44.164026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5755c2-49ad-40ca-8b0b-8974a633d9d4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:44.183459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1162/2353] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 08:57:44 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:57:44.188030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3498d-99cc-46bd-8882-98c21c626cf3 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:44.198436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3498d-99cc-46bd-8882-98c21c626cf3 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:44.203855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e3e5b-57c7-44ec-a053-3f7b835a98a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:44.204508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e3e5b-57c7-44ec-a053-3f7b835a98a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:44.204794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e3e5b-57c7-44ec-a053-3f7b835a98a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:44.205071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e3e5b-57c7-44ec-a053-3f7b835a98a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:44.205835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e3e5b-57c7-44ec-a053-3f7b835a98a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:44.207427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e3e5b-57c7-44ec-a053-3f7b835a98a2 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:44.217437 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1192/2354] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:57:44.221254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5755c2-49ad-40ca-8b0b-8974a633d9d4 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:44.231226 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5755c2-49ad-40ca-8b0b-8974a633d9d4 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:44.236002 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f993e-72ba-456f-87fd-62393be31b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:44.236644 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f993e-72ba-456f-87fd-62393be31b3d admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:44.236931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f993e-72ba-456f-87fd-62393be31b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:44.237258 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f993e-72ba-456f-87fd-62393be31b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:44.238011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f993e-72ba-456f-87fd-62393be31b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:44.239590 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f993e-72ba-456f-87fd-62393be31b3d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:44.250208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1163/2355] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 08:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:57:44.254231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e3e5b-57c7-44ec-a053-3f7b835a98a2 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:44.264319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e3e5b-57c7-44ec-a053-3f7b835a98a2 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:44.269572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707d029-3f09-4a84-9be6-f974adf26229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:44.270484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707d029-3f09-4a84-9be6-f974adf26229 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:44.270714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707d029-3f09-4a84-9be6-f974adf26229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:44.270988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707d029-3f09-4a84-9be6-f974adf26229 admin admin] PATH_INFO: `/v3/projects/72ac43cb053f4e68aa3c83e97a51cbbd/users/e47b6fa1c1684f449d21ff93e218ee84/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:44.286603 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707d029-3f09-4a84-9be6-f974adf26229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ac43cb053f4e68aa3c83e97a51cbbd, user_id=e47b6fa1c1684f449d21ff93e218ee84, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:44.289261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707d029-3f09-4a84-9be6-f974adf26229 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:44.302269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1193/2356] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 08:57:44 2022] PUT /identity/v3/projects/72ac43cb053f4e68aa3c83e97a51cbbd/users/e47b6fa1c1684f449d21ff93e218ee84/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:57:44.313230 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cde3d-8803-4fae-a232-2dc8883e84d8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:44.313521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cde3d-8803-4fae-a232-2dc8883e84d8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:44.313802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cde3d-8803-4fae-a232-2dc8883e84d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:44.340202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-cd2cde3d-8803-4fae-a232-2dc8883e84d8 None None] MFA Rules not processed for user `e47b6fa1c1684f449d21ff93e218ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:57:44.341693 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cd2cde3d-8803-4fae-a232-2dc8883e84d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:44.365023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:57:44.407253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1164/2357] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:57:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:57:44.417977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707d029-3f09-4a84-9be6-f974adf26229 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:44.419663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d707d029-3f09-4a84-9be6-f974adf26229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:44.476588 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707d029-3f09-4a84-9be6-f974adf26229 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:44.480208 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ee4ed-28b0-4b7d-af7d-dd4ccd521f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:44.480844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ee4ed-28b0-4b7d-af7d-dd4ccd521f1a service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:44.481154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ee4ed-28b0-4b7d-af7d-dd4ccd521f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:44.481435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ee4ed-28b0-4b7d-af7d-dd4ccd521f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:44.489688 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ee4ed-28b0-4b7d-af7d-dd4ccd521f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:44.491112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ee4ed-28b0-4b7d-af7d-dd4ccd521f1a service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:44.500336 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1194/2358] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:57:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:57:54.404247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=16 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:57:54.495958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:57:57.368256 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cde3d-8803-4fae-a232-2dc8883e84d8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:57.369704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cd2cde3d-8803-4fae-a232-2dc8883e84d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:57.457169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cde3d-8803-4fae-a232-2dc8883e84d8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:57.462940 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c39704-c105-483c-b394-a2a47acc876c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:57.463612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c39704-c105-483c-b394-a2a47acc876c service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:57.463986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c39704-c105-483c-b394-a2a47acc876c service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:57.464158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c39704-c105-483c-b394-a2a47acc876c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:57.466768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d5c39704-c105-483c-b394-a2a47acc876c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:57.482280 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c39704-c105-483c-b394-a2a47acc876c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:57.484777 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c39704-c105-483c-b394-a2a47acc876c service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:57.496897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1165/2359] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:57:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:57:59.306575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ee4ed-28b0-4b7d-af7d-dd4ccd521f1a service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:57:59.308235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ab0ee4ed-28b0-4b7d-af7d-dd4ccd521f1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:59.377074 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ee4ed-28b0-4b7d-af7d-dd4ccd521f1a service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:57:59.380924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6ed2a-d981-44e5-a80f-d71c2846c761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:57:59.381603 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6ed2a-d981-44e5-a80f-d71c2846c761 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:57:59.381891 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6ed2a-d981-44e5-a80f-d71c2846c761 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:57:59.382169 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6ed2a-d981-44e5-a80f-d71c2846c761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:57:59.383963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9cf6ed2a-d981-44e5-a80f-d71c2846c761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:57:59.401167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6ed2a-d981-44e5-a80f-d71c2846c761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:57:59.402619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6ed2a-d981-44e5-a80f-d71c2846c761 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:57:59.415305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1195/2360] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:57:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:58:07.491660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=34 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:58:09.409013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=26 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:58:52.429272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c39704-c105-483c-b394-a2a47acc876c service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:58:52.430703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d5c39704-c105-483c-b394-a2a47acc876c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:58:52.442820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c39704-c105-483c-b394-a2a47acc876c service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:58:52.446384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e24f9-a93e-468d-a0dc-faadc39e2b11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:58:52.447015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e24f9-a93e-468d-a0dc-faadc39e2b11 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:58:52.447304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e24f9-a93e-468d-a0dc-faadc39e2b11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:58:52.447584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e24f9-a93e-468d-a0dc-faadc39e2b11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:58:52.449335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e75e24f9-a93e-468d-a0dc-faadc39e2b11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:58:52.542261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e24f9-a93e-468d-a0dc-faadc39e2b11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:58:52.543712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e24f9-a93e-468d-a0dc-faadc39e2b11 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:58:52.563284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1166/2361] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:58:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:59:01.210028 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6ed2a-d981-44e5-a80f-d71c2846c761 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:59:01.242214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6ed2a-d981-44e5-a80f-d71c2846c761 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:59:01.248661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439af33c-92d4-4b5a-a677-5de5d9b3e813 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:59:01.249630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439af33c-92d4-4b5a-a677-5de5d9b3e813 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:59:01.250173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439af33c-92d4-4b5a-a677-5de5d9b3e813 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:59:01.250173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439af33c-92d4-4b5a-a677-5de5d9b3e813 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:59:01.252636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-439af33c-92d4-4b5a-a677-5de5d9b3e813 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:59:01.265740 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439af33c-92d4-4b5a-a677-5de5d9b3e813 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:59:01.267368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439af33c-92d4-4b5a-a677-5de5d9b3e813 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:59:01.281238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1196/2362] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:59:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:59:02.559471 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=47 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:59:03.036338 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33abeb8-f3b8-4a63-a642-54d31237d0b0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:59:03.036740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33abeb8-f3b8-4a63-a642-54d31237d0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:59:03.037109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33abeb8-f3b8-4a63-a642-54d31237d0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:59:03.087186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b33abeb8-f3b8-4a63-a642-54d31237d0b0 None None] MFA Rules not processed for user `4f603b289d964ee3ba3931948acf6788`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:59:03.088407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b33abeb8-f3b8-4a63-a642-54d31237d0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:59:03.154539 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1167/2363] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 08:59:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:59:03.452566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439af33c-92d4-4b5a-a677-5de5d9b3e813 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:59:03.454355 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-439af33c-92d4-4b5a-a677-5de5d9b3e813 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:59:03.466621 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439af33c-92d4-4b5a-a677-5de5d9b3e813 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:59:03.470231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da91ce9-e0a9-410a-8ce4-7c3d858afad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:59:03.470864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da91ce9-e0a9-410a-8ce4-7c3d858afad1 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:59:03.471150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da91ce9-e0a9-410a-8ce4-7c3d858afad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:59:03.471429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da91ce9-e0a9-410a-8ce4-7c3d858afad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:59:03.473160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4da91ce9-e0a9-410a-8ce4-7c3d858afad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:59:03.482503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da91ce9-e0a9-410a-8ce4-7c3d858afad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:59:03.483925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da91ce9-e0a9-410a-8ce4-7c3d858afad1 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:59:03.491844 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1197/2364] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:59:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:59:03.807975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33abeb8-f3b8-4a63-a642-54d31237d0b0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:59:03.810116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b33abeb8-f3b8-4a63-a642-54d31237d0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:59:03.922802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33abeb8-f3b8-4a63-a642-54d31237d0b0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:59:03.927856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089b027-b60e-4488-83cb-c7207a8c4bc2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:59:03.928502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089b027-b60e-4488-83cb-c7207a8c4bc2 service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:59:03.928788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089b027-b60e-4488-83cb-c7207a8c4bc2 service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:59:03.929060 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089b027-b60e-4488-83cb-c7207a8c4bc2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:59:03.930885 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5089b027-b60e-4488-83cb-c7207a8c4bc2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:59:03.943957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089b027-b60e-4488-83cb-c7207a8c4bc2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:59:03.945492 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089b027-b60e-4488-83cb-c7207a8c4bc2 service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:59:03.956178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1168/2365] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:59:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:59:13.059848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da91ce9-e0a9-410a-8ce4-7c3d858afad1 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:59:13.062200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4da91ce9-e0a9-410a-8ce4-7c3d858afad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:59:13.124317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da91ce9-e0a9-410a-8ce4-7c3d858afad1 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:59:13.130870 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddfc32-15d0-45bb-aad7-c7fbc2e68d56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:59:13.131550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddfc32-15d0-45bb-aad7-c7fbc2e68d56 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:59:13.131841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddfc32-15d0-45bb-aad7-c7fbc2e68d56 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:59:13.132124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddfc32-15d0-45bb-aad7-c7fbc2e68d56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:59:13.134007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a2ddfc32-15d0-45bb-aad7-c7fbc2e68d56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:59:13.146890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ddfc32-15d0-45bb-aad7-c7fbc2e68d56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:59:13.148628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ddfc32-15d0-45bb-aad7-c7fbc2e68d56 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:59:13.159260 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1198/2366] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 08:59:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:59:13.951888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=86 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:59:23.154759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=37 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:59:57.910385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d871e-9f18-4314-a705-915eb5798b33 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:59:57.910903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d871e-9f18-4314-a705-915eb5798b33 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:59:57.911014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d871e-9f18-4314-a705-915eb5798b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:59:57.970604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a43d871e-9f18-4314-a705-915eb5798b33 None None] MFA Rules not processed for user `26d6eb06b23b4db88261ae58d650736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:59:57.971741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a43d871e-9f18-4314-a705-915eb5798b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:59:58.034557 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1169/2367] 10.222.0.39 () {58 vars in 967 bytes} [Tue Jun 7 08:59:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:59:58.079874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddfc32-15d0-45bb-aad7-c7fbc2e68d56 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:59:58.081316 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a2ddfc32-15d0-45bb-aad7-c7fbc2e68d56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:59:58.170884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddfc32-15d0-45bb-aad7-c7fbc2e68d56 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:59:58.175856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f7b24-4f2d-4c78-8553-6c8480c558bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586ddffa77a54521b42b2aa50f16cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:59:58.176497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f7b24-4f2d-4c78-8553-6c8480c558bf service swift] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:59:58.176806 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f7b24-4f2d-4c78-8553-6c8480c558bf service swift] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:59:58.177099 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f7b24-4f2d-4c78-8553-6c8480c558bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:59:58.185702 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f7b24-4f2d-4c78-8553-6c8480c558bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:59:58.187276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f7b24-4f2d-4c78-8553-6c8480c558bf service swift] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:59:58.195182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1199/2368] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jun 7 08:59:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:00:08.031618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=44 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:00:08.192004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=41 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:00:52.906523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f27cd2-ff8d-434e-b6e5-e502294c1b8f None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:00:52.907015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f27cd2-ff8d-434e-b6e5-e502294c1b8f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:00:52.907145 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f27cd2-ff8d-434e-b6e5-e502294c1b8f None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:00:52.908445 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1170/2369] 10.222.0.39 () {54 vars in 912 bytes} [Tue Jun 7 09:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:00:52.913689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295885fa-7c2c-40ee-814a-99661dba1235 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:00:52.914165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295885fa-7c2c-40ee-814a-99661dba1235 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:00:52.914273 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295885fa-7c2c-40ee-814a-99661dba1235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:00:52.951133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-295885fa-7c2c-40ee-814a-99661dba1235 None None] MFA Rules not processed for user `7a7d1e8a0f4b4511a7dc8b27651744c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:00:52.952618 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-295885fa-7c2c-40ee-814a-99661dba1235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:00:52.961067 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1200/2370] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jun 7 09:00:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:00:52.983091 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f27cd2-ff8d-434e-b6e5-e502294c1b8f None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:00:52.985135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-59f27cd2-ff8d-434e-b6e5-e502294c1b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:00:53.003256 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f27cd2-ff8d-434e-b6e5-e502294c1b8f None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:00:53.008450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7350b-67b9-41b2-9976-f81dc30d308b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:00:53.009140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7350b-67b9-41b2-9976-f81dc30d308b service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:00:53.009431 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7350b-67b9-41b2-9976-f81dc30d308b service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:00:53.009712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7350b-67b9-41b2-9976-f81dc30d308b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:00:53.027903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7350b-67b9-41b2-9976-f81dc30d308b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:00:53.029538 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7350b-67b9-41b2-9976-f81dc30d308b service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:00:53.039471 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1171/2371] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:00:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:00:55.369550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08841dee-2617-4b9c-8bce-1b5dea4b6cf8 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:00:55.370048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08841dee-2617-4b9c-8bce-1b5dea4b6cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:00:55.370175 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08841dee-2617-4b9c-8bce-1b5dea4b6cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:00:55.396854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-08841dee-2617-4b9c-8bce-1b5dea4b6cf8 None None] MFA Rules not processed for user `876e3626cae145048dd1dbb4650a5603`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:00:55.398113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-08841dee-2617-4b9c-8bce-1b5dea4b6cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:00:55.452874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1201/2372] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:00:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:00:55.753535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7350b-67b9-41b2-9976-f81dc30d308b service placement] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:00:55.763383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7350b-67b9-41b2-9976-f81dc30d308b service placement] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:00:55.766948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e3a4f-82af-488b-b1ce-8f8c7d7da22b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:00:55.767578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e3a4f-82af-488b-b1ce-8f8c7d7da22b service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:00:55.767855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e3a4f-82af-488b-b1ce-8f8c7d7da22b service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:00:55.768126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e3a4f-82af-488b-b1ce-8f8c7d7da22b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:00:55.769877 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-956e3a4f-82af-488b-b1ce-8f8c7d7da22b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:00:55.780344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e3a4f-82af-488b-b1ce-8f8c7d7da22b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:00:55.781789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e3a4f-82af-488b-b1ce-8f8c7d7da22b service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:00:55.789579 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1172/2373] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:00:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:01:05.445519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=37 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:01:05.786129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=12 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:01:32.602224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08841dee-2617-4b9c-8bce-1b5dea4b6cf8 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:01:32.631184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08841dee-2617-4b9c-8bce-1b5dea4b6cf8 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:01:32.636068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5ce01-d6dd-49c8-97d2-bd63246108b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:01:32.636718 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5ce01-d6dd-49c8-97d2-bd63246108b3 service placement] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:01:32.636998 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5ce01-d6dd-49c8-97d2-bd63246108b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:01:32.637297 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5ce01-d6dd-49c8-97d2-bd63246108b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:01:32.639034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-28f5ce01-d6dd-49c8-97d2-bd63246108b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:01:32.662571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5ce01-d6dd-49c8-97d2-bd63246108b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:01:32.664148 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5ce01-d6dd-49c8-97d2-bd63246108b3 service placement] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:01:32.674510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1202/2374] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:01:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:01:32.996866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e3a4f-82af-488b-b1ce-8f8c7d7da22b service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:01:33.009820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e3a4f-82af-488b-b1ce-8f8c7d7da22b service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:01:33.015974 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77808c9-d41e-4abf-833d-2e7bed943b08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:01:33.016612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77808c9-d41e-4abf-833d-2e7bed943b08 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:01:33.016896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77808c9-d41e-4abf-833d-2e7bed943b08 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:01:33.017211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77808c9-d41e-4abf-833d-2e7bed943b08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:01:33.019081 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e77808c9-d41e-4abf-833d-2e7bed943b08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:01:33.098525 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77808c9-d41e-4abf-833d-2e7bed943b08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:01:33.100111 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77808c9-d41e-4abf-833d-2e7bed943b08 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:01:33.119722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1173/2375] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:01:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:01:42.670422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=18 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:01:43.035767 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5ce01-d6dd-49c8-97d2-bd63246108b3 service placement] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:01:43.055055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5ce01-d6dd-49c8-97d2-bd63246108b3 service placement] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:01:43.058804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682ab53-c5e8-4bed-9cbf-f374b2e62718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:01:43.059468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682ab53-c5e8-4bed-9cbf-f374b2e62718 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:01:43.059753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682ab53-c5e8-4bed-9cbf-f374b2e62718 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:01:43.060030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682ab53-c5e8-4bed-9cbf-f374b2e62718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:01:43.061826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9682ab53-c5e8-4bed-9cbf-f374b2e62718 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:01:43.081813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682ab53-c5e8-4bed-9cbf-f374b2e62718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:01:43.083235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682ab53-c5e8-4bed-9cbf-f374b2e62718 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:01:43.106377 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1203/2376] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:01:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:01:43.117132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=41 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:01:53.102584 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=17 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:01:57.911939 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77808c9-d41e-4abf-833d-2e7bed943b08 service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:01:57.931583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77808c9-d41e-4abf-833d-2e7bed943b08 service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:01:57.935474 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db44ee9-23d1-434e-8370-6fcd7fbf156f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:01:57.936113 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db44ee9-23d1-434e-8370-6fcd7fbf156f service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:01:57.936401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db44ee9-23d1-434e-8370-6fcd7fbf156f service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:01:57.936685 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db44ee9-23d1-434e-8370-6fcd7fbf156f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:01:57.938486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8db44ee9-23d1-434e-8370-6fcd7fbf156f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:01:58.018441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db44ee9-23d1-434e-8370-6fcd7fbf156f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:01:58.019851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db44ee9-23d1-434e-8370-6fcd7fbf156f service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:01:58.038624 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1174/2377] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:01:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:01:58.085725 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682ab53-c5e8-4bed-9cbf-f374b2e62718 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:01:58.095862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682ab53-c5e8-4bed-9cbf-f374b2e62718 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:01:58.099428 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01db9cd-391e-4652-bade-e293ca9455d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:01:58.100063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01db9cd-391e-4652-bade-e293ca9455d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:01:58.100350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01db9cd-391e-4652-bade-e293ca9455d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:01:58.100627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01db9cd-391e-4652-bade-e293ca9455d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:01:58.102406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d01db9cd-391e-4652-bade-e293ca9455d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:01:58.113348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01db9cd-391e-4652-bade-e293ca9455d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:01:58.114770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01db9cd-391e-4652-bade-e293ca9455d8 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:01:58.123891 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1204/2378] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:01:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:02:08.036010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=45 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:02:08.119516 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:02:32.278963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db44ee9-23d1-434e-8370-6fcd7fbf156f service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:02:32.288116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db44ee9-23d1-434e-8370-6fcd7fbf156f service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:02:32.291744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35f877-1762-4120-8b0e-11169a4ec914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:02:32.292382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35f877-1762-4120-8b0e-11169a4ec914 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:02:32.292670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35f877-1762-4120-8b0e-11169a4ec914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:02:32.292948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35f877-1762-4120-8b0e-11169a4ec914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:02:32.295353 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2d35f877-1762-4120-8b0e-11169a4ec914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:02:32.317794 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35f877-1762-4120-8b0e-11169a4ec914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:02:32.319219 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35f877-1762-4120-8b0e-11169a4ec914 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:02:32.327379 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1175/2379] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:02:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:02:42.323600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=11 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:02:43.084373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01db9cd-391e-4652-bade-e293ca9455d8 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:02:43.097947 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01db9cd-391e-4652-bade-e293ca9455d8 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:02:43.104585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25a5e0-e72a-421d-bd72-1ceb86e48f20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:02:43.105306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25a5e0-e72a-421d-bd72-1ceb86e48f20 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:02:43.105596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25a5e0-e72a-421d-bd72-1ceb86e48f20 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:02:43.105946 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25a5e0-e72a-421d-bd72-1ceb86e48f20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:02:43.107762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0b25a5e0-e72a-421d-bd72-1ceb86e48f20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:02:43.120395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25a5e0-e72a-421d-bd72-1ceb86e48f20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:02:43.122349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25a5e0-e72a-421d-bd72-1ceb86e48f20 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:02:43.131815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1205/2380] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:02:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:02:53.127571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:02:53.917336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35f877-1762-4120-8b0e-11169a4ec914 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:02:53.927210 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35f877-1762-4120-8b0e-11169a4ec914 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:02:53.931155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe77e4-7439-4e56-80fd-f932ec4d8801 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:02:53.931802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe77e4-7439-4e56-80fd-f932ec4d8801 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:02:53.932086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe77e4-7439-4e56-80fd-f932ec4d8801 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:02:53.932602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe77e4-7439-4e56-80fd-f932ec4d8801 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:02:53.941739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe77e4-7439-4e56-80fd-f932ec4d8801 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:02:53.943175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe77e4-7439-4e56-80fd-f932ec4d8801 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:02:53.952736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1176/2381] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:02:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:03:00.193195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25a5e0-e72a-421d-bd72-1ceb86e48f20 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:03:00.202615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25a5e0-e72a-421d-bd72-1ceb86e48f20 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:03:00.206234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0ed61-9443-47bf-bc32-b3958e39d794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:03:00.206871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0ed61-9443-47bf-bc32-b3958e39d794 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:03:00.207155 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0ed61-9443-47bf-bc32-b3958e39d794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:03:00.207432 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0ed61-9443-47bf-bc32-b3958e39d794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:03:00.218207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0ed61-9443-47bf-bc32-b3958e39d794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:03:00.219625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0ed61-9443-47bf-bc32-b3958e39d794 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:03:00.230253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1206/2382] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:03:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:03:03.947895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:03:10.225662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:03:58.668055 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738d817-8e1d-48c3-a16b-a53f5f9a0f3b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:03:58.668606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738d817-8e1d-48c3-a16b-a53f5f9a0f3b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:03:58.668606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738d817-8e1d-48c3-a16b-a53f5f9a0f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:03:58.695552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1738d817-8e1d-48c3-a16b-a53f5f9a0f3b None None] MFA Rules not processed for user `aa3150d932224bb0a737a3977fff46bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:03:58.696747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1738d817-8e1d-48c3-a16b-a53f5f9a0f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:03:58.703804 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1177/2383] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:03:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:04:05.924615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0ed61-9443-47bf-bc32-b3958e39d794 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:04:05.946043 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0ed61-9443-47bf-bc32-b3958e39d794 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:04:05.952706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ab3ca-64e4-4db8-934b-acf1e4e87cc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:04:05.953409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ab3ca-64e4-4db8-934b-acf1e4e87cc9 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:04:05.953701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ab3ca-64e4-4db8-934b-acf1e4e87cc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:04:05.953973 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ab3ca-64e4-4db8-934b-acf1e4e87cc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:04:05.972136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ab3ca-64e4-4db8-934b-acf1e4e87cc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:04:05.977405 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ab3ca-64e4-4db8-934b-acf1e4e87cc9 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:04:05.991165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1207/2384] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:04:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:04:08.693303 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:04:14.144289 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738d817-8e1d-48c3-a16b-a53f5f9a0f3b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:04:14.156991 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738d817-8e1d-48c3-a16b-a53f5f9a0f3b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:04:14.162520 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec434296-d7ea-4588-9483-1e7d1e03d94e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:04:14.163169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec434296-d7ea-4588-9483-1e7d1e03d94e service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:04:14.163456 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec434296-d7ea-4588-9483-1e7d1e03d94e service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:04:14.163735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec434296-d7ea-4588-9483-1e7d1e03d94e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:04:14.173333 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec434296-d7ea-4588-9483-1e7d1e03d94e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:04:14.174916 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec434296-d7ea-4588-9483-1e7d1e03d94e service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:04:14.186268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1178/2385] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:04:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:04:15.985668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:04:24.181110 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:04:54.102550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ab3ca-64e4-4db8-934b-acf1e4e87cc9 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:04:54.112161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ab3ca-64e4-4db8-934b-acf1e4e87cc9 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:04:54.115751 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2aa75-ab08-45da-9eb7-2dd3d6c1185e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:04:54.116377 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2aa75-ab08-45da-9eb7-2dd3d6c1185e service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:04:54.116663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2aa75-ab08-45da-9eb7-2dd3d6c1185e service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:04:54.116941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2aa75-ab08-45da-9eb7-2dd3d6c1185e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:04:54.125740 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2aa75-ab08-45da-9eb7-2dd3d6c1185e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:04:54.127159 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2aa75-ab08-45da-9eb7-2dd3d6c1185e service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:04:54.135831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1208/2386] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:04:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:05:04.131885 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:06:39.358223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec434296-d7ea-4588-9483-1e7d1e03d94e service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:06:39.369447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec434296-d7ea-4588-9483-1e7d1e03d94e service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:06:39.374400 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a93ee-3c7e-4bd3-9d8d-c447ef2ddc0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:06:39.375694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a93ee-3c7e-4bd3-9d8d-c447ef2ddc0c service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:06:39.376043 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a93ee-3c7e-4bd3-9d8d-c447ef2ddc0c service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:06:39.376326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a93ee-3c7e-4bd3-9d8d-c447ef2ddc0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:06:39.384761 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a93ee-3c7e-4bd3-9d8d-c447ef2ddc0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:06:39.386355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a93ee-3c7e-4bd3-9d8d-c447ef2ddc0c service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:06:39.396578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1179/2387] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:06:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:06:49.392461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:06:54.592987 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2aa75-ab08-45da-9eb7-2dd3d6c1185e service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:06:54.603707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2aa75-ab08-45da-9eb7-2dd3d6c1185e service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:06:54.607469 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab5a97-59c8-4cb0-913e-b904dbb129b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:06:54.608107 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab5a97-59c8-4cb0-913e-b904dbb129b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:06:54.608395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab5a97-59c8-4cb0-913e-b904dbb129b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:06:54.608793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab5a97-59c8-4cb0-913e-b904dbb129b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:06:54.619200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab5a97-59c8-4cb0-913e-b904dbb129b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:06:54.620937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab5a97-59c8-4cb0-913e-b904dbb129b4 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:06:54.630851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1209/2388] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:06:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:07:04.626647 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:40.273017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a93ee-3c7e-4bd3-9d8d-c447ef2ddc0c service placement] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:40.283214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a93ee-3c7e-4bd3-9d8d-c447ef2ddc0c service placement] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:40.286810 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d0332-78a6-4c4d-af61-3933019671ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:40.287442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0332-78a6-4c4d-af61-3933019671ca service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:40.287728 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0332-78a6-4c4d-af61-3933019671ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:40.288004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0332-78a6-4c4d-af61-3933019671ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:40.295189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d0332-78a6-4c4d-af61-3933019671ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:40.297350 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d0332-78a6-4c4d-af61-3933019671ca service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:40.305215 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1180/2389] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:07:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:07:50.301752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:07:58.697769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab5a97-59c8-4cb0-913e-b904dbb129b4 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:58.699377 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c2ab5a97-59c8-4cb0-913e-b904dbb129b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:58.766557 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab5a97-59c8-4cb0-913e-b904dbb129b4 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:58.771142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a5460-14d8-4373-9317-726871213e60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:58.771798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a5460-14d8-4373-9317-726871213e60 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.773183 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a5460-14d8-4373-9317-726871213e60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.773183 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a5460-14d8-4373-9317-726871213e60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.783651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a5460-14d8-4373-9317-726871213e60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:58.785111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a5460-14d8-4373-9317-726871213e60 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:58.814528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1210/2390] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:07:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:07:58.856126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d0332-78a6-4c4d-af61-3933019671ca service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:58.865923 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d0332-78a6-4c4d-af61-3933019671ca service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:58.869524 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d375ea-1dc6-4f88-bf56-1b0b10d9b76d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:58.870152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d375ea-1dc6-4f88-bf56-1b0b10d9b76d service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.870434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d375ea-1dc6-4f88-bf56-1b0b10d9b76d service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.870707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d375ea-1dc6-4f88-bf56-1b0b10d9b76d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.878994 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d375ea-1dc6-4f88-bf56-1b0b10d9b76d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:58.880420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d375ea-1dc6-4f88-bf56-1b0b10d9b76d service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:58.889911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1181/2391] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:07:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:07:58.939009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a5460-14d8-4373-9317-726871213e60 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:58.947359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a5460-14d8-4373-9317-726871213e60 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:58.950962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afae03f-c163-4b2b-b968-0b4a4e94bcaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:58.951717 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afae03f-c163-4b2b-b968-0b4a4e94bcaf service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:58.951999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afae03f-c163-4b2b-b968-0b4a4e94bcaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:58.952274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afae03f-c163-4b2b-b968-0b4a4e94bcaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:58.960827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afae03f-c163-4b2b-b968-0b4a4e94bcaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:58.962283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afae03f-c163-4b2b-b968-0b4a4e94bcaf service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:58.971821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1211/2392] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:07:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:07:59.033452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d375ea-1dc6-4f88-bf56-1b0b10d9b76d service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:07:59.041814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d375ea-1dc6-4f88-bf56-1b0b10d9b76d service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:07:59.045367 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9cb0b-3061-4e75-97f0-099e716903e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:07:59.046002 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9cb0b-3061-4e75-97f0-099e716903e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:07:59.046916 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9cb0b-3061-4e75-97f0-099e716903e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:07:59.047415 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9cb0b-3061-4e75-97f0-099e716903e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:07:59.049294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8df9cb0b-3061-4e75-97f0-099e716903e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:07:59.132689 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9cb0b-3061-4e75-97f0-099e716903e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:07:59.134169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9cb0b-3061-4e75-97f0-099e716903e2 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:07:59.152458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1182/2393] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:07:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:08:08.966994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=44 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:08:09.149954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=47 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:09:59.342881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afae03f-c163-4b2b-b968-0b4a4e94bcaf service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:09:59.361483 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afae03f-c163-4b2b-b968-0b4a4e94bcaf service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:09:59.365141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de06d72-cefa-4b05-8925-c36c38b7e488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:09:59.365777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de06d72-cefa-4b05-8925-c36c38b7e488 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:09:59.366062 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de06d72-cefa-4b05-8925-c36c38b7e488 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:09:59.366334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de06d72-cefa-4b05-8925-c36c38b7e488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:09:59.368205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-3de06d72-cefa-4b05-8925-c36c38b7e488 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:09:59.469795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de06d72-cefa-4b05-8925-c36c38b7e488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:09:59.471152 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de06d72-cefa-4b05-8925-c36c38b7e488 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:09:59.488974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1212/2394] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:09:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:10:09.486409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=56 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:10:57.523271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9cb0b-3061-4e75-97f0-099e716903e2 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:57.524913 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8df9cb0b-3061-4e75-97f0-099e716903e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:10:57.625985 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9cb0b-3061-4e75-97f0-099e716903e2 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:57.630166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7200c8-ce01-48a7-a286-3556bbe97376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:57.630865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7200c8-ce01-48a7-a286-3556bbe97376 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:57.631171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7200c8-ce01-48a7-a286-3556bbe97376 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:57.631419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7200c8-ce01-48a7-a286-3556bbe97376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:57.634083 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bf7200c8-ce01-48a7-a286-3556bbe97376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:10:57.642134 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de06d72-cefa-4b05-8925-c36c38b7e488 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:57.644009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-3de06d72-cefa-4b05-8925-c36c38b7e488 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:10:57.734344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7200c8-ce01-48a7-a286-3556bbe97376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:57.735689 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7200c8-ce01-48a7-a286-3556bbe97376 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:57.746791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de06d72-cefa-4b05-8925-c36c38b7e488 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:57.747715 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1183/2395] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:10:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:10:57.752424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d2b25-3cf3-4184-be78-8a070ad77ec5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:57.753066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d2b25-3cf3-4184-be78-8a070ad77ec5 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:57.753376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d2b25-3cf3-4184-be78-8a070ad77ec5 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:57.753649 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d2b25-3cf3-4184-be78-8a070ad77ec5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:57.756204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-543d2b25-3cf3-4184-be78-8a070ad77ec5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:10:57.774026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d2b25-3cf3-4184-be78-8a070ad77ec5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:57.775859 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d2b25-3cf3-4184-be78-8a070ad77ec5 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:57.787834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1213/2396] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:10:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:10:58.930348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7200c8-ce01-48a7-a286-3556bbe97376 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:10:58.931900 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bf7200c8-ce01-48a7-a286-3556bbe97376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:10:58.943147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7200c8-ce01-48a7-a286-3556bbe97376 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:10:58.946738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e040b-27c1-42ff-8dc3-662da346a38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:10:58.947371 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e040b-27c1-42ff-8dc3-662da346a38e service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:10:58.947656 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e040b-27c1-42ff-8dc3-662da346a38e service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:10:58.947935 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e040b-27c1-42ff-8dc3-662da346a38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:10:58.958933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e040b-27c1-42ff-8dc3-662da346a38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:10:58.960337 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e040b-27c1-42ff-8dc3-662da346a38e service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:10:58.971289 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1184/2397] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:10:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:01.373641 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788519e-fa09-4c57-a0e5-334309582eb1 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:01.374119 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788519e-fa09-4c57-a0e5-334309582eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:01.374321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788519e-fa09-4c57-a0e5-334309582eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:01.416518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-f788519e-fa09-4c57-a0e5-334309582eb1 None None] MFA Rules not processed for user `5caffe8082c246c7b00f1eaa98154c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:01.418517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f788519e-fa09-4c57-a0e5-334309582eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:01.439894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1214/2398] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:01.502360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e040b-27c1-42ff-8dc3-662da346a38e service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:01.518291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e040b-27c1-42ff-8dc3-662da346a38e service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:01.526573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2db4f-a3f9-4def-aaf1-726fcc6c4395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:01.529877 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2db4f-a3f9-4def-aaf1-726fcc6c4395 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:01.530440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2db4f-a3f9-4def-aaf1-726fcc6c4395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:01.530885 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2db4f-a3f9-4def-aaf1-726fcc6c4395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:01.534126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-05f2db4f-a3f9-4def-aaf1-726fcc6c4395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:01.608889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:01.663712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2db4f-a3f9-4def-aaf1-726fcc6c4395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:01.665416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2db4f-a3f9-4def-aaf1-726fcc6c4395 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:01.692694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1185/2399] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:01.915614 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788519e-fa09-4c57-a0e5-334309582eb1 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:01.926490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788519e-fa09-4c57-a0e5-334309582eb1 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:01.930255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c4607-fc7b-4b53-b262-aa1f1849977c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:01.931011 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c4607-fc7b-4b53-b262-aa1f1849977c service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:01.931366 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c4607-fc7b-4b53-b262-aa1f1849977c service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:01.931715 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c4607-fc7b-4b53-b262-aa1f1849977c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:01.934258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9b1c4607-fc7b-4b53-b262-aa1f1849977c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:01.944378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c4607-fc7b-4b53-b262-aa1f1849977c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:01.945932 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c4607-fc7b-4b53-b262-aa1f1849977c service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:01.954864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1215/2400] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:02.468258 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2db4f-a3f9-4def-aaf1-726fcc6c4395 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:02.470385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-05f2db4f-a3f9-4def-aaf1-726fcc6c4395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:02.499598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2db4f-a3f9-4def-aaf1-726fcc6c4395 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:02.515744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:02.516747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:02.517129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:02.517490 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] PATH_INFO: `/v3/users/e47b6fa1c1684f449d21ff93e218ee84` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:02.518812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e47b6fa1c1684f449d21ff93e218ee84)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:02.520812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:02.625890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e47b6fa1c1684f449d21ff93e218ee84'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:02.626981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e47b6fa1c1684f449d21ff93e218ee84'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:02.663852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e47b6fa1c1684f449d21ff93e218ee84'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:02.667621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e47b6fa1c1684f449d21ff93e218ee84'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:02.754610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e47b6fa1c1684f449d21ff93e218ee84'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:02.789146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1186/2401] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:02 2022] DELETE /identity/v3/users/e47b6fa1c1684f449d21ff93e218ee84 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:02.793567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65c135-8ee5-4bc2-83aa-4681c8e13505 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:02.794205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65c135-8ee5-4bc2-83aa-4681c8e13505 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:02.794520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65c135-8ee5-4bc2-83aa-4681c8e13505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:02.845402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-de65c135-8ee5-4bc2-83aa-4681c8e13505 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:02.857434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-de65c135-8ee5-4bc2-83aa-4681c8e13505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:02.971151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1216/2402] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:02.988937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:02.997506 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65c135-8ee5-4bc2-83aa-4681c8e13505 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:02.998922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-de65c135-8ee5-4bc2-83aa-4681c8e13505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:03.000614 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:03.100813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:03.127991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65c135-8ee5-4bc2-83aa-4681c8e13505 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:03.142236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d223c-b380-43a5-9490-5ed478436e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:03.143401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d223c-b380-43a5-9490-5ed478436e04 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:03.144569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d223c-b380-43a5-9490-5ed478436e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:03.145238 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822ee47-45de-4f8e-9602-16930670eaf1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:03.145643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d223c-b380-43a5-9490-5ed478436e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:03.152503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11957b-0fd7-4cba-8934-7933a61258de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:03.154895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11957b-0fd7-4cba-8934-7933a61258de service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:03.155187 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11957b-0fd7-4cba-8934-7933a61258de service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:03.155463 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11957b-0fd7-4cba-8934-7933a61258de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:03.157475 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9c11957b-0fd7-4cba-8934-7933a61258de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:03.169791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d223c-b380-43a5-9490-5ed478436e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:03.171545 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d223c-b380-43a5-9490-5ed478436e04 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:03.174808 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11957b-0fd7-4cba-8934-7933a61258de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:03.176429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11957b-0fd7-4cba-8934-7933a61258de service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:03.195840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1217/2403] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:11:03.202939 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1187/2404] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:04.128280 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d223c-b380-43a5-9490-5ed478436e04 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:04.129958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-db7d223c-b380-43a5-9490-5ed478436e04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:04.161499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d223c-b380-43a5-9490-5ed478436e04 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:04.177852 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:04.178937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:04.179374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:04.180394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] PATH_INFO: `/v3/projects/72ac43cb053f4e68aa3c83e97a51cbbd` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:04.183420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] RBAC: Authorizing `identity:delete_project(project_id=72ac43cb053f4e68aa3c83e97a51cbbd)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:04.185911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:04.287521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72ac43cb053f4e68aa3c83e97a51cbbd', 'request_id': 'req-7678a59f-49f7-4b55-a809-9ee32663a64e'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:04.296087 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72ac43cb053f4e68aa3c83e97a51cbbd', 'request_id': 'req-7678a59f-49f7-4b55-a809-9ee32663a64e'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:04.317169 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72ac43cb053f4e68aa3c83e97a51cbbd', 'request_id': 'req-7678a59f-49f7-4b55-a809-9ee32663a64e'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:04.329274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] The token cache is being invalidate because project 72ac43cb053f4e68aa3c83e97a51cbbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:04.329605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:04.330222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:04.332735 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1218/2405] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:04 2022] DELETE /identity/v3/projects/72ac43cb053f4e68aa3c83e97a51cbbd => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:07.314173 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11957b-0fd7-4cba-8934-7933a61258de service placement] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:07.315606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9c11957b-0fd7-4cba-8934-7933a61258de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:07.338565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:07.419684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11957b-0fd7-4cba-8934-7933a61258de service placement] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:07.546274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e886c1-4ee2-4ff1-8130-cfb2bc5345df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:07.547231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e886c1-4ee2-4ff1-8130-cfb2bc5345df service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:07.547516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e886c1-4ee2-4ff1-8130-cfb2bc5345df service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:07.547790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e886c1-4ee2-4ff1-8130-cfb2bc5345df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:07.549659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f8e886c1-4ee2-4ff1-8130-cfb2bc5345df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:07.563427 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e886c1-4ee2-4ff1-8130-cfb2bc5345df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:07.565015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e886c1-4ee2-4ff1-8130-cfb2bc5345df service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:07.578925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1188/2406] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:07.693213 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:07.694904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:07.779490 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e886c1-4ee2-4ff1-8130-cfb2bc5345df service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:07.791151 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f8e886c1-4ee2-4ff1-8130-cfb2bc5345df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:07.822306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678a59f-49f7-4b55-a809-9ee32663a64e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:07.828114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4ed6b-1f0e-49a8-9058-fa721b0af7fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:07.828861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4ed6b-1f0e-49a8-9058-fa721b0af7fb service placement] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:07.829274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4ed6b-1f0e-49a8-9058-fa721b0af7fb service placement] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:07.829556 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4ed6b-1f0e-49a8-9058-fa721b0af7fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:07.833451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e6d4ed6b-1f0e-49a8-9058-fa721b0af7fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:07.884254 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4ed6b-1f0e-49a8-9058-fa721b0af7fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:07.890931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4ed6b-1f0e-49a8-9058-fa721b0af7fb service placement] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:07.914444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e886c1-4ee2-4ff1-8130-cfb2bc5345df service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:07.916937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1219/2407] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:07.931287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e5ae1-d729-4987-bb75-28ff04e00fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:07.932963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e5ae1-d729-4987-bb75-28ff04e00fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:07.933305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e5ae1-d729-4987-bb75-28ff04e00fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:07.934480 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e5ae1-d729-4987-bb75-28ff04e00fe8 admin admin] PATH_INFO: `/v3/users/876e3626cae145048dd1dbb4650a5603` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:07.973412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e5ae1-d729-4987-bb75-28ff04e00fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=876e3626cae145048dd1dbb4650a5603)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:07.975411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e5ae1-d729-4987-bb75-28ff04e00fe8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:08.020502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:08.054865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b78e5ae1-d729-4987-bb75-28ff04e00fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '876e3626cae145048dd1dbb4650a5603'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:08.055769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b78e5ae1-d729-4987-bb75-28ff04e00fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '876e3626cae145048dd1dbb4650a5603'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:08.068193 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b78e5ae1-d729-4987-bb75-28ff04e00fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '876e3626cae145048dd1dbb4650a5603'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:08.068828 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b78e5ae1-d729-4987-bb75-28ff04e00fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '876e3626cae145048dd1dbb4650a5603'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:08.084548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b78e5ae1-d729-4987-bb75-28ff04e00fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '876e3626cae145048dd1dbb4650a5603'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:08.100164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1189/2408] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:07 2022] DELETE /identity/v3/users/876e3626cae145048dd1dbb4650a5603 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:11:08.345755 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4ed6b-1f0e-49a8-9058-fa721b0af7fb service placement] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:08.358443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4ed6b-1f0e-49a8-9058-fa721b0af7fb service placement] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:08.441884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:08.450181 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff39b5f-9e72-4799-a35a-2599342414b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:08.455672 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff39b5f-9e72-4799-a35a-2599342414b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:08.456370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff39b5f-9e72-4799-a35a-2599342414b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:08.456933 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff39b5f-9e72-4799-a35a-2599342414b3 admin admin] PATH_INFO: `/v3/projects/cc6d788a2d5d4a5c9f559836207e8d62` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:08.459708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff39b5f-9e72-4799-a35a-2599342414b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc6d788a2d5d4a5c9f559836207e8d62)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:08.461680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff39b5f-9e72-4799-a35a-2599342414b3 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:08.523034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0ff39b5f-9e72-4799-a35a-2599342414b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc6d788a2d5d4a5c9f559836207e8d62', 'request_id': 'req-0ff39b5f-9e72-4799-a35a-2599342414b3'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:08.533597 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0ff39b5f-9e72-4799-a35a-2599342414b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc6d788a2d5d4a5c9f559836207e8d62', 'request_id': 'req-0ff39b5f-9e72-4799-a35a-2599342414b3'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:08.544092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0ff39b5f-9e72-4799-a35a-2599342414b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc6d788a2d5d4a5c9f559836207e8d62', 'request_id': 'req-0ff39b5f-9e72-4799-a35a-2599342414b3'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:08.559749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0ff39b5f-9e72-4799-a35a-2599342414b3 admin admin] The token cache is being invalidate because project cc6d788a2d5d4a5c9f559836207e8d62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:08.560137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0ff39b5f-9e72-4799-a35a-2599342414b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:08.561022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0ff39b5f-9e72-4799-a35a-2599342414b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:08.562802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1220/2409] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:08 2022] DELETE /identity/v3/projects/cc6d788a2d5d4a5c9f559836207e8d62 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:08.569962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd539b70-6fb6-4266-8b2c-de05062161e7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:08.570505 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd539b70-6fb6-4266-8b2c-de05062161e7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:08.570505 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd539b70-6fb6-4266-8b2c-de05062161e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:08.640428 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-fd539b70-6fb6-4266-8b2c-de05062161e7 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:08.641708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fd539b70-6fb6-4266-8b2c-de05062161e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:08.717867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1190/2410] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:08.726000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb58df0-801c-4e06-8689-2cdf6eecd06b None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:08.726504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb58df0-801c-4e06-8689-2cdf6eecd06b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:08.726637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb58df0-801c-4e06-8689-2cdf6eecd06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:08.748681 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-cdb58df0-801c-4e06-8689-2cdf6eecd06b None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:08.749955 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-cdb58df0-801c-4e06-8689-2cdf6eecd06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:08.759242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1221/2411] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:08.764420 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd539b70-6fb6-4266-8b2c-de05062161e7 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:08.776743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd539b70-6fb6-4266-8b2c-de05062161e7 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:08.782582 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e9dd0-5687-4718-a9ca-7081b8630ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:08.783229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e9dd0-5687-4718-a9ca-7081b8630ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:08.783529 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e9dd0-5687-4718-a9ca-7081b8630ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:08.784439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e9dd0-5687-4718-a9ca-7081b8630ae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:08.785261 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e9dd0-5687-4718-a9ca-7081b8630ae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:08.785568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e9dd0-5687-4718-a9ca-7081b8630ae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:08.787173 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e9dd0-5687-4718-a9ca-7081b8630ae3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:08.806544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1191/2412] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:08.811109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb58df0-801c-4e06-8689-2cdf6eecd06b None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:08.825310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb58df0-801c-4e06-8689-2cdf6eecd06b None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:08.833401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f2404-28f0-41a4-9f6d-87832576dd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:08.833401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f2404-28f0-41a4-9f6d-87832576dd39 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:08.833401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f2404-28f0-41a4-9f6d-87832576dd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:08.833401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f2404-28f0-41a4-9f6d-87832576dd39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:08.834980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f2404-28f0-41a4-9f6d-87832576dd39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:08.836176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f2404-28f0-41a4-9f6d-87832576dd39 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:08.852166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e9dd0-5687-4718-a9ca-7081b8630ae3 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:08.854368 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f20e9dd0-5687-4718-a9ca-7081b8630ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:08.901413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1222/2413] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:08 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:08.906421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f2404-28f0-41a4-9f6d-87832576dd39 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:08.974735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e9dd0-5687-4718-a9ca-7081b8630ae3 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:09.006463 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f2404-28f0-41a4-9f6d-87832576dd39 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:09.011857 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107ba10-3ef5-4466-b68b-4b9a2726028b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:09.012965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107ba10-3ef5-4466-b68b-4b9a2726028b admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:09.013375 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107ba10-3ef5-4466-b68b-4b9a2726028b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:09.013662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107ba10-3ef5-4466-b68b-4b9a2726028b admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:09.014369 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107ba10-3ef5-4466-b68b-4b9a2726028b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:09.016044 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:09.016330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107ba10-3ef5-4466-b68b-4b9a2726028b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:09.053249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1223/2414] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:08 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:09.056158 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107ba10-3ef5-4466-b68b-4b9a2726028b admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:09.060573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d68ea7-8d64-40c8-9e1d-047cab79e5dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:09.061282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d68ea7-8d64-40c8-9e1d-047cab79e5dd service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:09.061573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d68ea7-8d64-40c8-9e1d-047cab79e5dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:09.061854 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d68ea7-8d64-40c8-9e1d-047cab79e5dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:09.064392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-68d68ea7-8d64-40c8-9e1d-047cab79e5dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:09.075509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107ba10-3ef5-4466-b68b-4b9a2726028b admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:09.079118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d68ea7-8d64-40c8-9e1d-047cab79e5dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:09.081974 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d68ea7-8d64-40c8-9e1d-047cab79e5dd service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:09.089316 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb466bf0-9fcd-4d3d-801b-404d0471e618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:09.090390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb466bf0-9fcd-4d3d-801b-404d0471e618 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:09.090980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb466bf0-9fcd-4d3d-801b-404d0471e618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:09.091114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb466bf0-9fcd-4d3d-801b-404d0471e618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:09.092008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb466bf0-9fcd-4d3d-801b-404d0471e618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:09.094041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb466bf0-9fcd-4d3d-801b-404d0471e618 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:09.096336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1192/2415] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:09.107901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1224/2416] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:09.113375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d68ea7-8d64-40c8-9e1d-047cab79e5dd service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:09.125485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d68ea7-8d64-40c8-9e1d-047cab79e5dd service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:09.130550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7e8a9-e9b7-4458-ad17-5b5ebfb3f750 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:09.131000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7e8a9-e9b7-4458-ad17-5b5ebfb3f750 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:09.131389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439988d4-2eff-4b19-bc43-0723b43f6ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:09.131734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7e8a9-e9b7-4458-ad17-5b5ebfb3f750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:09.132082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439988d4-2eff-4b19-bc43-0723b43f6ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:09.132565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439988d4-2eff-4b19-bc43-0723b43f6ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:09.133041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439988d4-2eff-4b19-bc43-0723b43f6ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:09.134385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439988d4-2eff-4b19-bc43-0723b43f6ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:09.136372 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439988d4-2eff-4b19-bc43-0723b43f6ceb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:09.149426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1193/2417] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:09.154080 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439988d4-2eff-4b19-bc43-0723b43f6ceb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:09.165068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439988d4-2eff-4b19-bc43-0723b43f6ceb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:09.170663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d015a-36dd-454c-bd1c-0cdfbbcc4c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:09.171554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d015a-36dd-454c-bd1c-0cdfbbcc4c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:09.171837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d015a-36dd-454c-bd1c-0cdfbbcc4c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:09.172112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d015a-36dd-454c-bd1c-0cdfbbcc4c08 admin admin] PATH_INFO: `/v3/projects/0ae9b9141a0b443cb26fddbcf60d36f6/users/48470725d9cf44de84590585deddcc31/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:09.179997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:09.188604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d015a-36dd-454c-bd1c-0cdfbbcc4c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae9b9141a0b443cb26fddbcf60d36f6, user_id=48470725d9cf44de84590585deddcc31, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:09.192041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d015a-36dd-454c-bd1c-0cdfbbcc4c08 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:09.194114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-f0d7e8a9-e9b7-4458-ad17-5b5ebfb3f750 None None] MFA Rules not processed for user `4f603b289d964ee3ba3931948acf6788`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:09.196047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f0d7e8a9-e9b7-4458-ad17-5b5ebfb3f750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:09.209120 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1194/2418] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:09 2022] PUT /identity/v3/projects/0ae9b9141a0b443cb26fddbcf60d36f6/users/48470725d9cf44de84590585deddcc31/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:09.272242 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e7f74-455a-46f8-8501-3f8eef08ef26 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:09.272534 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e7f74-455a-46f8-8501-3f8eef08ef26 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:09.272806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e7f74-455a-46f8-8501-3f8eef08ef26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:09.276610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1225/2419] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:09.311401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2c7e7f74-455a-46f8-8501-3f8eef08ef26 None None] MFA Rules not processed for user `48470725d9cf44de84590585deddcc31`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:09.312651 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2c7e7f74-455a-46f8-8501-3f8eef08ef26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:09.396202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1195/2420] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:11:09.406808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7e8a9-e9b7-4458-ad17-5b5ebfb3f750 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:09.408119 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f0d7e8a9-e9b7-4458-ad17-5b5ebfb3f750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:09.488624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7e8a9-e9b7-4458-ad17-5b5ebfb3f750 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:09.575076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f5f2c-5e9f-4f36-afaf-15bb35f8bd97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:09.575908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f5f2c-5e9f-4f36-afaf-15bb35f8bd97 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:09.576300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f5f2c-5e9f-4f36-afaf-15bb35f8bd97 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:09.576797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f5f2c-5e9f-4f36-afaf-15bb35f8bd97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:09.588112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f5f2c-5e9f-4f36-afaf-15bb35f8bd97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:09.590859 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f5f2c-5e9f-4f36-afaf-15bb35f8bd97 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:09.602647 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1226/2421] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:09.855164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e7f74-455a-46f8-8501-3f8eef08ef26 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:09.953879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e7f74-455a-46f8-8501-3f8eef08ef26 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:09.960622 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a081b6d-eb00-4622-953d-25fa4013ab44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:09.961346 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a081b6d-eb00-4622-953d-25fa4013ab44 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:09.961636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a081b6d-eb00-4622-953d-25fa4013ab44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:09.961914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a081b6d-eb00-4622-953d-25fa4013ab44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:09.966509 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4a081b6d-eb00-4622-953d-25fa4013ab44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:09.985229 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a081b6d-eb00-4622-953d-25fa4013ab44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:09.988645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a081b6d-eb00-4622-953d-25fa4013ab44 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:10.012859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1196/2422] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:12.838018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f5f2c-5e9f-4f36-afaf-15bb35f8bd97 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:12.896838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:12.956032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f5f2c-5e9f-4f36-afaf-15bb35f8bd97 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:12.962794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:12.964096 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:12.964709 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:12.965364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] PATH_INFO: `/v3/users/48470725d9cf44de84590585deddcc31` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:12.967394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=48470725d9cf44de84590585deddcc31)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:12.970597 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:13.064316 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48470725d9cf44de84590585deddcc31'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:13.065437 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48470725d9cf44de84590585deddcc31'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:13.074589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48470725d9cf44de84590585deddcc31'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:13.090256 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48470725d9cf44de84590585deddcc31'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:13.091152 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48470725d9cf44de84590585deddcc31'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:13.120342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1227/2423] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:12 2022] DELETE /identity/v3/users/48470725d9cf44de84590585deddcc31 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:13.126523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821e179-5cbf-4e08-b920-d9fd4e225b84 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:13.126926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821e179-5cbf-4e08-b920-d9fd4e225b84 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:13.127077 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821e179-5cbf-4e08-b920-d9fd4e225b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:13.146254 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4821e179-5cbf-4e08-b920-d9fd4e225b84 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:13.147468 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4821e179-5cbf-4e08-b920-d9fd4e225b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:13.182566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:13.270517 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1197/2424] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:13.283478 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:13.285008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:13.419058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506394a-7a00-47ca-84be-621c21e2b3ec admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:13.423719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c4f0b-3f0a-4729-87da-818d190c2f15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:13.424345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c4f0b-3f0a-4729-87da-818d190c2f15 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:13.424640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c4f0b-3f0a-4729-87da-818d190c2f15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:13.425048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c4f0b-3f0a-4729-87da-818d190c2f15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:13.437154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c4f0b-3f0a-4729-87da-818d190c2f15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:13.438454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c4f0b-3f0a-4729-87da-818d190c2f15 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:13.451096 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1228/2425] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:13.751215 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821e179-5cbf-4e08-b920-d9fd4e225b84 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:13.768662 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4821e179-5cbf-4e08-b920-d9fd4e225b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:13.782802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821e179-5cbf-4e08-b920-d9fd4e225b84 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:13.788456 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:13.789375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:13.789657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:13.789929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] PATH_INFO: `/v3/users/3e6fe6fa3fc14cca97064b49403eb68b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:13.791116 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e6fe6fa3fc14cca97064b49403eb68b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:13.793689 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:13.862225 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e6fe6fa3fc14cca97064b49403eb68b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:13.863308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e6fe6fa3fc14cca97064b49403eb68b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:13.883076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e6fe6fa3fc14cca97064b49403eb68b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:13.883762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e6fe6fa3fc14cca97064b49403eb68b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:13.906066 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e6fe6fa3fc14cca97064b49403eb68b'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:13.926934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1198/2426] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:13 2022] DELETE /identity/v3/users/3e6fe6fa3fc14cca97064b49403eb68b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:14.082066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c4f0b-3f0a-4729-87da-818d190c2f15 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.084372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9e5c4f0b-3f0a-4729-87da-818d190c2f15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:14.098655 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c4f0b-3f0a-4729-87da-818d190c2f15 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.162905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:14.193452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be082ef2-6257-4136-a460-15ca7b10ea03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.194316 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be082ef2-6257-4136-a460-15ca7b10ea03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.194604 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be082ef2-6257-4136-a460-15ca7b10ea03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.194882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be082ef2-6257-4136-a460-15ca7b10ea03 admin admin] PATH_INFO: `/v3/projects/0ae9b9141a0b443cb26fddbcf60d36f6` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.196089 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be082ef2-6257-4136-a460-15ca7b10ea03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae9b9141a0b443cb26fddbcf60d36f6)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.198114 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be082ef2-6257-4136-a460-15ca7b10ea03 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.225629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.228457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:14.243176 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8bd75-9858-4eb3-b317-e128915ec31d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.258994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-be082ef2-6257-4136-a460-15ca7b10ea03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae9b9141a0b443cb26fddbcf60d36f6', 'request_id': 'req-be082ef2-6257-4136-a460-15ca7b10ea03'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:14.267934 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-be082ef2-6257-4136-a460-15ca7b10ea03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae9b9141a0b443cb26fddbcf60d36f6', 'request_id': 'req-be082ef2-6257-4136-a460-15ca7b10ea03'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:14.280700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-be082ef2-6257-4136-a460-15ca7b10ea03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae9b9141a0b443cb26fddbcf60d36f6', 'request_id': 'req-be082ef2-6257-4136-a460-15ca7b10ea03'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:14.308207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-be082ef2-6257-4136-a460-15ca7b10ea03 admin admin] The token cache is being invalidate because project 0ae9b9141a0b443cb26fddbcf60d36f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:14.308802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-be082ef2-6257-4136-a460-15ca7b10ea03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:14.309705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-be082ef2-6257-4136-a460-15ca7b10ea03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:14.311740 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1229/2427] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:14 2022] DELETE /identity/v3/projects/0ae9b9141a0b443cb26fddbcf60d36f6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:14.316324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc1f3b-ed82-476a-893f-d1a128ce2273 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.316596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc1f3b-ed82-476a-893f-d1a128ce2273 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.316840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc1f3b-ed82-476a-893f-d1a128ce2273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.319540 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4964ec-bff9-4856-b2ea-3facad6de58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.320518 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4964ec-bff9-4856-b2ea-3facad6de58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.320879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4964ec-bff9-4856-b2ea-3facad6de58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.321258 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4964ec-bff9-4856-b2ea-3facad6de58d admin admin] PATH_INFO: `/v3/projects/e7bbcea61e7549fbac7aecd75f14c316` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.322652 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4964ec-bff9-4856-b2ea-3facad6de58d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7bbcea61e7549fbac7aecd75f14c316)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.325616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4964ec-bff9-4856-b2ea-3facad6de58d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.338457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-07cc1f3b-ed82-476a-893f-d1a128ce2273 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:14.339959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-07cc1f3b-ed82-476a-893f-d1a128ce2273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:14.349758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1230/2428] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:14.357937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ce8a2-85ee-4f48-8f4e-b7212c2ef95d None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.358263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ce8a2-85ee-4f48-8f4e-b7212c2ef95d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.358555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ce8a2-85ee-4f48-8f4e-b7212c2ef95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.375011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:14.388489 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a4964ec-bff9-4856-b2ea-3facad6de58d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7bbcea61e7549fbac7aecd75f14c316', 'request_id': 'req-0a4964ec-bff9-4856-b2ea-3facad6de58d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:14.390781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-2f1ce8a2-85ee-4f48-8f4e-b7212c2ef95d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:14.391778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-2f1ce8a2-85ee-4f48-8f4e-b7212c2ef95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:14.401446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a4964ec-bff9-4856-b2ea-3facad6de58d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7bbcea61e7549fbac7aecd75f14c316', 'request_id': 'req-0a4964ec-bff9-4856-b2ea-3facad6de58d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:14.427472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a4964ec-bff9-4856-b2ea-3facad6de58d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7bbcea61e7549fbac7aecd75f14c316', 'request_id': 'req-0a4964ec-bff9-4856-b2ea-3facad6de58d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:14.444894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a4964ec-bff9-4856-b2ea-3facad6de58d admin admin] The token cache is being invalidate because project e7bbcea61e7549fbac7aecd75f14c316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:14.445227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a4964ec-bff9-4856-b2ea-3facad6de58d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:14.445921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0a4964ec-bff9-4856-b2ea-3facad6de58d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:14.448255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1199/2429] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:14 2022] DELETE /identity/v3/projects/e7bbcea61e7549fbac7aecd75f14c316 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:11:14.454554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee21ca-4787-467e-96ae-6769dfaa7ea7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.454898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee21ca-4787-467e-96ae-6769dfaa7ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.455172 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee21ca-4787-467e-96ae-6769dfaa7ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.476176 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a9ee21ca-4787-467e-96ae-6769dfaa7ea7 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:14.477432 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a9ee21ca-4787-467e-96ae-6769dfaa7ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:14.487527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1231/2430] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:14.491737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ce8a2-85ee-4f48-8f4e-b7212c2ef95d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.501479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ce8a2-85ee-4f48-8f4e-b7212c2ef95d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.506263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbeeea-5e46-4011-aa97-05c72734dd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.506907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbeeea-5e46-4011-aa97-05c72734dd46 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.507194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbeeea-5e46-4011-aa97-05c72734dd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.507471 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbeeea-5e46-4011-aa97-05c72734dd46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.508185 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbeeea-5e46-4011-aa97-05c72734dd46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:14.508498 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbeeea-5e46-4011-aa97-05c72734dd46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.510115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbeeea-5e46-4011-aa97-05c72734dd46 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.525891 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1232/2431] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:11:14.526792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbeeea-5e46-4011-aa97-05c72734dd46 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.536380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbeeea-5e46-4011-aa97-05c72734dd46 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.538801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1200/2432] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:11:14.541476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be611b-d8e9-4a0b-8420-7962929fef5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.542117 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be611b-d8e9-4a0b-8420-7962929fef5a admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.542397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be611b-d8e9-4a0b-8420-7962929fef5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.542815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be611b-d8e9-4a0b-8420-7962929fef5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.543546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be611b-d8e9-4a0b-8420-7962929fef5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.545463 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be611b-d8e9-4a0b-8420-7962929fef5a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.546266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6e214-94a5-47f5-b275-20fe58d988e1 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.546563 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6e214-94a5-47f5-b275-20fe58d988e1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.546852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6e214-94a5-47f5-b275-20fe58d988e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.567225 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-56a6e214-94a5-47f5-b275-20fe58d988e1 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:14.568564 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-56a6e214-94a5-47f5-b275-20fe58d988e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:14.580826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1233/2433] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:14 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:14.581021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1201/2434] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:11:14.582574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6e214-94a5-47f5-b275-20fe58d988e1 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.582700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be611b-d8e9-4a0b-8420-7962929fef5a admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.593015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6e214-94a5-47f5-b275-20fe58d988e1 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.618450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:14.651864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be611b-d8e9-4a0b-8420-7962929fef5a admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.657033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2848f8-0756-4ce2-974a-2765a39bd852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.657233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371ce41-8031-4d66-8a56-71df54923113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.657741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2848f8-0756-4ce2-974a-2765a39bd852 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.657872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371ce41-8031-4d66-8a56-71df54923113 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.658027 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2848f8-0756-4ce2-974a-2765a39bd852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.658159 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371ce41-8031-4d66-8a56-71df54923113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.658311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2848f8-0756-4ce2-974a-2765a39bd852 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.658440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371ce41-8031-4d66-8a56-71df54923113 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.658999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2848f8-0756-4ce2-974a-2765a39bd852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.659167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371ce41-8031-4d66-8a56-71df54923113 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:14.659444 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371ce41-8031-4d66-8a56-71df54923113 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.660938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2848f8-0756-4ce2-974a-2765a39bd852 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.661050 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371ce41-8031-4d66-8a56-71df54923113 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.673353 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1234/2435] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:11:14.678936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371ce41-8031-4d66-8a56-71df54923113 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.684690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1202/2436] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:14 2022] POST /identity/v3/users => generated 281 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:14.686291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2848f8-0756-4ce2-974a-2765a39bd852 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.688737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371ce41-8031-4d66-8a56-71df54923113 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.693386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55ddd3-163f-4c8d-9c3e-bd682809ec30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.694023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55ddd3-163f-4c8d-9c3e-bd682809ec30 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.694311 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55ddd3-163f-4c8d-9c3e-bd682809ec30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.694590 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55ddd3-163f-4c8d-9c3e-bd682809ec30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.695299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55ddd3-163f-4c8d-9c3e-bd682809ec30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.695826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2848f8-0756-4ce2-974a-2765a39bd852 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.697231 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55ddd3-163f-4c8d-9c3e-bd682809ec30 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.700766 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba98a0f-4588-4fcb-a11e-34791288c536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.701448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba98a0f-4588-4fcb-a11e-34791288c536 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.701735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba98a0f-4588-4fcb-a11e-34791288c536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.702013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba98a0f-4588-4fcb-a11e-34791288c536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.702727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba98a0f-4588-4fcb-a11e-34791288c536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.704299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba98a0f-4588-4fcb-a11e-34791288c536 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.714171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1203/2437] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:11:14.718642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba98a0f-4588-4fcb-a11e-34791288c536 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.725219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1235/2438] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:14 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:14.728430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba98a0f-4588-4fcb-a11e-34791288c536 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.728816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55ddd3-163f-4c8d-9c3e-bd682809ec30 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.768721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:14.788347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc97f52-d871-4c24-baf4-5763c47c93e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.789065 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc97f52-d871-4c24-baf4-5763c47c93e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.789385 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc97f52-d871-4c24-baf4-5763c47c93e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.789673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc97f52-d871-4c24-baf4-5763c47c93e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.790386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc97f52-d871-4c24-baf4-5763c47c93e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.791975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc97f52-d871-4c24-baf4-5763c47c93e6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.795115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55ddd3-163f-4c8d-9c3e-bd682809ec30 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.799945 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef921d53-4e70-4ac8-9dbf-1ca5fed82a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.800607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef921d53-4e70-4ac8-9dbf-1ca5fed82a57 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.800924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef921d53-4e70-4ac8-9dbf-1ca5fed82a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.801242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef921d53-4e70-4ac8-9dbf-1ca5fed82a57 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.801958 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef921d53-4e70-4ac8-9dbf-1ca5fed82a57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.805266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1204/2439] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:11:14.805395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef921d53-4e70-4ac8-9dbf-1ca5fed82a57 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.806188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc97f52-d871-4c24-baf4-5763c47c93e6 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.816549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc97f52-d871-4c24-baf4-5763c47c93e6 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.821549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9227dda-842b-4857-9ab1-c87164959aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.822373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9227dda-842b-4857-9ab1-c87164959aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.822656 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9227dda-842b-4857-9ab1-c87164959aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.822933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9227dda-842b-4857-9ab1-c87164959aa3 admin admin] PATH_INFO: `/v3/projects/46509eb17dc6461c87e39747569181b1/users/1dab88dbce5247f5991309917123e324/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.828163 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1236/2440] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:14 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:14.832197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef921d53-4e70-4ac8-9dbf-1ca5fed82a57 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.839756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9227dda-842b-4857-9ab1-c87164959aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46509eb17dc6461c87e39747569181b1, user_id=1dab88dbce5247f5991309917123e324, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.842526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9227dda-842b-4857-9ab1-c87164959aa3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.842639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef921d53-4e70-4ac8-9dbf-1ca5fed82a57 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.848126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e0728-1c50-4122-95aa-d3a74bb178ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.849113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e0728-1c50-4122-95aa-d3a74bb178ba admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.849637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e0728-1c50-4122-95aa-d3a74bb178ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.850005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e0728-1c50-4122-95aa-d3a74bb178ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.851177 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e0728-1c50-4122-95aa-d3a74bb178ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.853326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e0728-1c50-4122-95aa-d3a74bb178ba admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.858789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1205/2441] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:14 2022] PUT /identity/v3/projects/46509eb17dc6461c87e39747569181b1/users/1dab88dbce5247f5991309917123e324/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:14.866914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1237/2442] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:11:14.868226 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9227dda-842b-4857-9ab1-c87164959aa3 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.868600 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563bd75-350c-4a2b-bbb9-203c4512a608 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.868888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563bd75-350c-4a2b-bbb9-203c4512a608 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.869204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563bd75-350c-4a2b-bbb9-203c4512a608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.898290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1563bd75-350c-4a2b-bbb9-203c4512a608 None None] MFA Rules not processed for user `1dab88dbce5247f5991309917123e324`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:14.898988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1563bd75-350c-4a2b-bbb9-203c4512a608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:14.936315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9227dda-842b-4857-9ab1-c87164959aa3 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.941053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d4998-2f63-4d93-9560-efb3f9c4cb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.941740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d4998-2f63-4d93-9560-efb3f9c4cb6d admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.942023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d4998-2f63-4d93-9560-efb3f9c4cb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.942295 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d4998-2f63-4d93-9560-efb3f9c4cb6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.943005 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d4998-2f63-4d93-9560-efb3f9c4cb6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.944586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d4998-2f63-4d93-9560-efb3f9c4cb6d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.954815 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1206/2443] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:14.958802 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d4998-2f63-4d93-9560-efb3f9c4cb6d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.964342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1238/2444] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:11:14.968791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d4998-2f63-4d93-9560-efb3f9c4cb6d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.969178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563bd75-350c-4a2b-bbb9-203c4512a608 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:14.973729 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db21ab0-e6a9-4274-9767-a9bc0fc021d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.974590 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db21ab0-e6a9-4274-9767-a9bc0fc021d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.974865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db21ab0-e6a9-4274-9767-a9bc0fc021d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.975139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db21ab0-e6a9-4274-9767-a9bc0fc021d2 admin admin] PATH_INFO: `/v3/projects/2ffa5b55381f4f1b9676af05dbddff73/users/1c8d9f15a131430d92060bea3cd13c96/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.981426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563bd75-350c-4a2b-bbb9-203c4512a608 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:14.986199 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73232a9-83e7-4eaa-852a-f9c604fdf5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:14.986854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73232a9-83e7-4eaa-852a-f9c604fdf5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:14.987418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73232a9-83e7-4eaa-852a-f9c604fdf5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:14.987418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73232a9-83e7-4eaa-852a-f9c604fdf5a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:14.988098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73232a9-83e7-4eaa-852a-f9c604fdf5a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.990052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db21ab0-e6a9-4274-9767-a9bc0fc021d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffa5b55381f4f1b9676af05dbddff73, user_id=1c8d9f15a131430d92060bea3cd13c96, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:14.990166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73232a9-83e7-4eaa-852a-f9c604fdf5a1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:14.992660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db21ab0-e6a9-4274-9767-a9bc0fc021d2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:15.002115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:15.007053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1207/2445] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:14 2022] PUT /identity/v3/projects/2ffa5b55381f4f1b9676af05dbddff73/users/1c8d9f15a131430d92060bea3cd13c96/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:15.020086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882b273-e6d3-43dc-9756-935dbff968d9 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:15.020380 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882b273-e6d3-43dc-9756-935dbff968d9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:15.020674 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882b273-e6d3-43dc-9756-935dbff968d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:15.020819 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1239/2446] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:14 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:15.024464 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73232a9-83e7-4eaa-852a-f9c604fdf5a1 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:15.047871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5882b273-e6d3-43dc-9756-935dbff968d9 None None] MFA Rules not processed for user `1c8d9f15a131430d92060bea3cd13c96`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:15.049114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5882b273-e6d3-43dc-9756-935dbff968d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:15.088543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73232a9-83e7-4eaa-852a-f9c604fdf5a1 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:15.094305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62700dd2-af19-4e78-85e7-32baeeac2ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:15.095218 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62700dd2-af19-4e78-85e7-32baeeac2ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:15.095511 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62700dd2-af19-4e78-85e7-32baeeac2ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:15.095791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62700dd2-af19-4e78-85e7-32baeeac2ec9 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:15.096474 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62700dd2-af19-4e78-85e7-32baeeac2ec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:15.098434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62700dd2-af19-4e78-85e7-32baeeac2ec9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:15.115443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1208/2447] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:11:15.118546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1240/2448] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:15 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:15.122497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882b273-e6d3-43dc-9756-935dbff968d9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:15.126442 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62700dd2-af19-4e78-85e7-32baeeac2ec9 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:15.128130 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-62700dd2-af19-4e78-85e7-32baeeac2ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:15.133397 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882b273-e6d3-43dc-9756-935dbff968d9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:15.138712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8a914-7c73-4a94-afff-1fa50368b506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:15.139368 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8a914-7c73-4a94-afff-1fa50368b506 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:15.139650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8a914-7c73-4a94-afff-1fa50368b506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:15.139924 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8a914-7c73-4a94-afff-1fa50368b506 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:15.140663 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8a914-7c73-4a94-afff-1fa50368b506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:15.142271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8a914-7c73-4a94-afff-1fa50368b506 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:15.152790 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1209/2449] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:15.156931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8a914-7c73-4a94-afff-1fa50368b506 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:15.162754 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:15.166978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8a914-7c73-4a94-afff-1fa50368b506 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:15.173677 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb16848-24da-4a9f-9c00-8a3e9adc62c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:15.174575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb16848-24da-4a9f-9c00-8a3e9adc62c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:15.174857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb16848-24da-4a9f-9c00-8a3e9adc62c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:15.175143 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb16848-24da-4a9f-9c00-8a3e9adc62c1 admin admin] PATH_INFO: `/v3/projects/839ccec442e94f5b80a1423a2c0c8fa5/users/a481a4f06cac409e90d6ec3393fb3917/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:15.188394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62700dd2-af19-4e78-85e7-32baeeac2ec9 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:15.193479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c2431-52d7-4aad-8f9d-2ab4eaffaa01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:15.194167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c2431-52d7-4aad-8f9d-2ab4eaffaa01 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:15.194274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb16848-24da-4a9f-9c00-8a3e9adc62c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=839ccec442e94f5b80a1423a2c0c8fa5, user_id=a481a4f06cac409e90d6ec3393fb3917, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:15.194507 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c2431-52d7-4aad-8f9d-2ab4eaffaa01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:15.194790 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c2431-52d7-4aad-8f9d-2ab4eaffaa01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:15.196993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb16848-24da-4a9f-9c00-8a3e9adc62c1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:15.205285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c2431-52d7-4aad-8f9d-2ab4eaffaa01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:15.207047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c2431-52d7-4aad-8f9d-2ab4eaffaa01 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:15.216452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1210/2450] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:15 2022] PUT /identity/v3/projects/839ccec442e94f5b80a1423a2c0c8fa5/users/a481a4f06cac409e90d6ec3393fb3917/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:11:15.217760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb16848-24da-4a9f-9c00-8a3e9adc62c1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:15.221581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1241/2451] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:11:15.286003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb16848-24da-4a9f-9c00-8a3e9adc62c1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:15.291763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673267a7-bea7-4814-bf3c-9c558e5dbfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:15.292431 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673267a7-bea7-4814-bf3c-9c558e5dbfe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:15.293496 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673267a7-bea7-4814-bf3c-9c558e5dbfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:15.293966 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673267a7-bea7-4814-bf3c-9c558e5dbfe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:15.294722 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673267a7-bea7-4814-bf3c-9c558e5dbfe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:15.296330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673267a7-bea7-4814-bf3c-9c558e5dbfe2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:15.317782 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1211/2452] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:15.322089 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c2431-52d7-4aad-8f9d-2ab4eaffaa01 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:15.334299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c2431-52d7-4aad-8f9d-2ab4eaffaa01 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:15.341482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d643d71-17f3-4ec7-ace9-0beddb6d52c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:15.342411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d643d71-17f3-4ec7-ace9-0beddb6d52c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:15.342701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d643d71-17f3-4ec7-ace9-0beddb6d52c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:15.342985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d643d71-17f3-4ec7-ace9-0beddb6d52c8 admin admin] PATH_INFO: `/v3/projects/839ccec442e94f5b80a1423a2c0c8fa5/users/a481a4f06cac409e90d6ec3393fb3917/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:15.346044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d643d71-17f3-4ec7-ace9-0beddb6d52c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=839ccec442e94f5b80a1423a2c0c8fa5, user_id=a481a4f06cac409e90d6ec3393fb3917, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:15.348839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d643d71-17f3-4ec7-ace9-0beddb6d52c8 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:15.366877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1242/2453] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:15 2022] PUT /identity/v3/projects/839ccec442e94f5b80a1423a2c0c8fa5/users/a481a4f06cac409e90d6ec3393fb3917/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:15.379612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb26dd-9883-4131-bdd1-61ec02623ca8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:15.379906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb26dd-9883-4131-bdd1-61ec02623ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:15.380188 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb26dd-9883-4131-bdd1-61ec02623ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:15.423648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-8fcb26dd-9883-4131-bdd1-61ec02623ca8 None None] MFA Rules not processed for user `a481a4f06cac409e90d6ec3393fb3917`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:15.425719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8fcb26dd-9883-4131-bdd1-61ec02623ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:15.530780 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1212/2454] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:15.544051 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d643d71-17f3-4ec7-ace9-0beddb6d52c8 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:15.546128 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9d643d71-17f3-4ec7-ace9-0beddb6d52c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:15.618335 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d643d71-17f3-4ec7-ace9-0beddb6d52c8 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:15.622664 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc99cf8-6739-4ec5-b533-84a8ac5ad03a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:15.623319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc99cf8-6739-4ec5-b533-84a8ac5ad03a service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:15.623599 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc99cf8-6739-4ec5-b533-84a8ac5ad03a service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:15.623872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc99cf8-6739-4ec5-b533-84a8ac5ad03a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:15.635785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc99cf8-6739-4ec5-b533-84a8ac5ad03a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:15.637391 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc99cf8-6739-4ec5-b533-84a8ac5ad03a service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:15.648653 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1243/2455] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:16.080992 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb26dd-9883-4131-bdd1-61ec02623ca8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:16.146739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb26dd-9883-4131-bdd1-61ec02623ca8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:16.151576 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:16.152500 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:16.153059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:16.153373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] PATH_INFO: `/v3/users/1dab88dbce5247f5991309917123e324` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:16.154567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dab88dbce5247f5991309917123e324)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:16.156473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:16.158884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:16.204364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dab88dbce5247f5991309917123e324'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:16.205076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dab88dbce5247f5991309917123e324'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:16.212969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dab88dbce5247f5991309917123e324'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:16.213653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dab88dbce5247f5991309917123e324'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:16.227673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dab88dbce5247f5991309917123e324'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:16.240855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1213/2456] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:16 2022] DELETE /identity/v3/users/1dab88dbce5247f5991309917123e324 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:16.244960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d72c4-5165-4391-b53f-3e53c9d54497 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:16.245443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d72c4-5165-4391-b53f-3e53c9d54497 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:16.245527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d72c4-5165-4391-b53f-3e53c9d54497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:16.263061 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-9a4d72c4-5165-4391-b53f-3e53c9d54497 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:16.264266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9a4d72c4-5165-4391-b53f-3e53c9d54497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:16.272973 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:16.326360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1244/2457] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:16.337442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:16.338932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:16.398719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93462911-848b-4845-9248-6c957cee5d48 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:16.403049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3b259-7321-4e20-b239-1d14c3d2b44f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:16.403894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3b259-7321-4e20-b239-1d14c3d2b44f service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:16.404180 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3b259-7321-4e20-b239-1d14c3d2b44f service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:16.404458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3b259-7321-4e20-b239-1d14c3d2b44f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:16.413970 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3b259-7321-4e20-b239-1d14c3d2b44f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:16.415402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3b259-7321-4e20-b239-1d14c3d2b44f service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:16.425531 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1214/2458] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:16.730429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d72c4-5165-4391-b53f-3e53c9d54497 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:16.732210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9a4d72c4-5165-4391-b53f-3e53c9d54497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:16.746778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d72c4-5165-4391-b53f-3e53c9d54497 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:16.751589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:16.752484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:16.752772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:16.753048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] PATH_INFO: `/v3/projects/46509eb17dc6461c87e39747569181b1` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:16.754279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46509eb17dc6461c87e39747569181b1)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:16.756176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:16.820301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46509eb17dc6461c87e39747569181b1', 'request_id': 'req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:16.828669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46509eb17dc6461c87e39747569181b1', 'request_id': 'req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:16.836685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46509eb17dc6461c87e39747569181b1', 'request_id': 'req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:16.850586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] The token cache is being invalidate because project 46509eb17dc6461c87e39747569181b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:16.851012 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:16.852367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:16.856874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1245/2459] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:16 2022] DELETE /identity/v3/projects/46509eb17dc6461c87e39747569181b1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:16.858928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3b259-7321-4e20-b239-1d14c3d2b44f service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:16.879339 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3b259-7321-4e20-b239-1d14c3d2b44f service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:16.970282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3ffa9-cd9f-4d8b-83fe-0fa17c5e8d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:16.971158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3ffa9-cd9f-4d8b-83fe-0fa17c5e8d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:16.971442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3ffa9-cd9f-4d8b-83fe-0fa17c5e8d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:16.971841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3ffa9-cd9f-4d8b-83fe-0fa17c5e8d10 admin admin] PATH_INFO: `/v3/users/a481a4f06cac409e90d6ec3393fb3917` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:16.973112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3ffa9-cd9f-4d8b-83fe-0fa17c5e8d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a481a4f06cac409e90d6ec3393fb3917)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:16.975029 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3ffa9-cd9f-4d8b-83fe-0fa17c5e8d10 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:17.009364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:17.037233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4ba3ffa9-cd9f-4d8b-83fe-0fa17c5e8d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a481a4f06cac409e90d6ec3393fb3917'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:17.037953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4ba3ffa9-cd9f-4d8b-83fe-0fa17c5e8d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a481a4f06cac409e90d6ec3393fb3917'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:17.047149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4ba3ffa9-cd9f-4d8b-83fe-0fa17c5e8d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a481a4f06cac409e90d6ec3393fb3917'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:17.047890 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4ba3ffa9-cd9f-4d8b-83fe-0fa17c5e8d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a481a4f06cac409e90d6ec3393fb3917'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:17.060190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-4ba3ffa9-cd9f-4d8b-83fe-0fa17c5e8d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a481a4f06cac409e90d6ec3393fb3917'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:17.073298 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1215/2460] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:16 2022] DELETE /identity/v3/users/a481a4f06cac409e90d6ec3393fb3917 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:17.385884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:17.388176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:17.408816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca3eab-f23b-4e10-85f3-3fd4f3abbd10 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:17.481226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870035a-2ffa-4da4-ba66-2ea7098fe883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:17.482220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870035a-2ffa-4da4-ba66-2ea7098fe883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:17.482500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870035a-2ffa-4da4-ba66-2ea7098fe883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:17.482777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870035a-2ffa-4da4-ba66-2ea7098fe883 admin admin] PATH_INFO: `/v3/projects/839ccec442e94f5b80a1423a2c0c8fa5` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:17.483949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870035a-2ffa-4da4-ba66-2ea7098fe883 admin admin] RBAC: Authorizing `identity:delete_project(project_id=839ccec442e94f5b80a1423a2c0c8fa5)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:17.485895 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870035a-2ffa-4da4-ba66-2ea7098fe883 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:17.524500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:17.533587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-f870035a-2ffa-4da4-ba66-2ea7098fe883 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '839ccec442e94f5b80a1423a2c0c8fa5', 'request_id': 'req-f870035a-2ffa-4da4-ba66-2ea7098fe883'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:17.540170 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-f870035a-2ffa-4da4-ba66-2ea7098fe883 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '839ccec442e94f5b80a1423a2c0c8fa5', 'request_id': 'req-f870035a-2ffa-4da4-ba66-2ea7098fe883'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:17.547785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-f870035a-2ffa-4da4-ba66-2ea7098fe883 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '839ccec442e94f5b80a1423a2c0c8fa5', 'request_id': 'req-f870035a-2ffa-4da4-ba66-2ea7098fe883'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:17.557975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-f870035a-2ffa-4da4-ba66-2ea7098fe883 admin admin] The token cache is being invalidate because project 839ccec442e94f5b80a1423a2c0c8fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:17.558326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-f870035a-2ffa-4da4-ba66-2ea7098fe883 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:17.559192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-f870035a-2ffa-4da4-ba66-2ea7098fe883 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:17.560780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1246/2461] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:17 2022] DELETE /identity/v3/projects/839ccec442e94f5b80a1423a2c0c8fa5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:17.565655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a432f19-5df1-46fa-9ce3-1ae26b54ad2a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:17.566158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a432f19-5df1-46fa-9ce3-1ae26b54ad2a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:17.566247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a432f19-5df1-46fa-9ce3-1ae26b54ad2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:17.587568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9a432f19-5df1-46fa-9ce3-1ae26b54ad2a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:17.588930 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9a432f19-5df1-46fa-9ce3-1ae26b54ad2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:17.658700 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1216/2462] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:17.669769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add02a31-a1be-4902-b23a-95d84cd362ed None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:17.670190 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add02a31-a1be-4902-b23a-95d84cd362ed None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:17.670586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add02a31-a1be-4902-b23a-95d84cd362ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:17.691299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-add02a31-a1be-4902-b23a-95d84cd362ed None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:17.692517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-add02a31-a1be-4902-b23a-95d84cd362ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:17.700656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1247/2463] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:17.704901 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a432f19-5df1-46fa-9ce3-1ae26b54ad2a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:17.715095 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a432f19-5df1-46fa-9ce3-1ae26b54ad2a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:17.719882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586ad7e-b3ef-4906-93e5-f6bc17e70ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:17.720794 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586ad7e-b3ef-4906-93e5-f6bc17e70ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:17.721100 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586ad7e-b3ef-4906-93e5-f6bc17e70ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:17.721384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586ad7e-b3ef-4906-93e5-f6bc17e70ef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:17.722113 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586ad7e-b3ef-4906-93e5-f6bc17e70ef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:17.722423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586ad7e-b3ef-4906-93e5-f6bc17e70ef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:17.724030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586ad7e-b3ef-4906-93e5-f6bc17e70ef4 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:17.737512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1217/2464] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:17.741730 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add02a31-a1be-4902-b23a-95d84cd362ed None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:17.750978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add02a31-a1be-4902-b23a-95d84cd362ed None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:17.755689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60828765-a9c7-4baa-b759-7bb373e6e7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:17.756343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60828765-a9c7-4baa-b759-7bb373e6e7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:17.756630 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60828765-a9c7-4baa-b759-7bb373e6e7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:17.757013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60828765-a9c7-4baa-b759-7bb373e6e7e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:17.757724 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60828765-a9c7-4baa-b759-7bb373e6e7e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:17.759546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60828765-a9c7-4baa-b759-7bb373e6e7e1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:17.789221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1248/2465] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:17 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:17.793852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586ad7e-b3ef-4906-93e5-f6bc17e70ef4 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:17.803693 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586ad7e-b3ef-4906-93e5-f6bc17e70ef4 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:17.860058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f668234-74c3-4e54-bd5d-929f5b7ce968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:17.860729 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f668234-74c3-4e54-bd5d-929f5b7ce968 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:17.861050 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f668234-74c3-4e54-bd5d-929f5b7ce968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:17.861362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f668234-74c3-4e54-bd5d-929f5b7ce968 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:17.862053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f668234-74c3-4e54-bd5d-929f5b7ce968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:17.863969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f668234-74c3-4e54-bd5d-929f5b7ce968 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:17.884933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1218/2466] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:17 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:17.890005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60828765-a9c7-4baa-b759-7bb373e6e7e1 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:17.906145 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60828765-a9c7-4baa-b759-7bb373e6e7e1 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:17.911116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edb2b9-994a-406e-8135-a158e1926d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:17.911835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edb2b9-994a-406e-8135-a158e1926d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:17.912124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edb2b9-994a-406e-8135-a158e1926d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:17.912403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edb2b9-994a-406e-8135-a158e1926d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:17.914418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edb2b9-994a-406e-8135-a158e1926d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:17.914778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edb2b9-994a-406e-8135-a158e1926d52 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:17.929521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1249/2467] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:17.930746 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f668234-74c3-4e54-bd5d-929f5b7ce968 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:17.942513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f668234-74c3-4e54-bd5d-929f5b7ce968 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:17.947460 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e712031-c6eb-4b20-98cc-b9d28dc62cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:17.948186 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e712031-c6eb-4b20-98cc-b9d28dc62cdb admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:17.948476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e712031-c6eb-4b20-98cc-b9d28dc62cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:17.949330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e712031-c6eb-4b20-98cc-b9d28dc62cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:17.950321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e712031-c6eb-4b20-98cc-b9d28dc62cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:17.951978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e712031-c6eb-4b20-98cc-b9d28dc62cdb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:17.965989 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1219/2468] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:17.969938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edb2b9-994a-406e-8135-a158e1926d52 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:17.981814 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edb2b9-994a-406e-8135-a158e1926d52 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:17.986960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7cfec-1470-4bf8-8e5c-31bac048d91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:17.987920 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7cfec-1470-4bf8-8e5c-31bac048d91d admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:17.988203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7cfec-1470-4bf8-8e5c-31bac048d91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:17.988477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7cfec-1470-4bf8-8e5c-31bac048d91d admin admin] PATH_INFO: `/v3/projects/eda8f759205b43e4859e6251411eee7d/users/5662bdf849904077a21d3b0e34afe5a5/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:18.004153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7cfec-1470-4bf8-8e5c-31bac048d91d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda8f759205b43e4859e6251411eee7d, user_id=5662bdf849904077a21d3b0e34afe5a5, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:18.006838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7cfec-1470-4bf8-8e5c-31bac048d91d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:18.020949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1250/2469] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:17 2022] PUT /identity/v3/projects/eda8f759205b43e4859e6251411eee7d/users/5662bdf849904077a21d3b0e34afe5a5/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:11:18.031622 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aacb61-f089-4add-82d4-0a306ccccb4c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:18.031933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aacb61-f089-4add-82d4-0a306ccccb4c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:18.032199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aacb61-f089-4add-82d4-0a306ccccb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:18.051339 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:18.062373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e3aacb61-f089-4add-82d4-0a306ccccb4c None None] MFA Rules not processed for user `5662bdf849904077a21d3b0e34afe5a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:18.064192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e3aacb61-f089-4add-82d4-0a306ccccb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:18.128596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1220/2470] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:18.133006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7cfec-1470-4bf8-8e5c-31bac048d91d admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:18.194293 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7cfec-1470-4bf8-8e5c-31bac048d91d admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:18.199443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7e2b3-577c-435e-a3d3-6ffcc3603bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:18.200090 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7e2b3-577c-435e-a3d3-6ffcc3603bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:18.200377 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7e2b3-577c-435e-a3d3-6ffcc3603bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:18.200660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7e2b3-577c-435e-a3d3-6ffcc3603bb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:18.201373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7e2b3-577c-435e-a3d3-6ffcc3603bb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:18.203278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7e2b3-577c-435e-a3d3-6ffcc3603bb5 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:18.229969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1251/2471] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:18 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:18.234894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aacb61-f089-4add-82d4-0a306ccccb4c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:18.246490 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aacb61-f089-4add-82d4-0a306ccccb4c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:18.304481 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8af542-612a-460a-95b6-f6fb6de3694b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:18.305191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8af542-612a-460a-95b6-f6fb6de3694b admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:18.305482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8af542-612a-460a-95b6-f6fb6de3694b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:18.305765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8af542-612a-460a-95b6-f6fb6de3694b admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:18.306452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8af542-612a-460a-95b6-f6fb6de3694b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:18.309108 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8af542-612a-460a-95b6-f6fb6de3694b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:18.330166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1221/2472] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:18 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:18.334194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7e2b3-577c-435e-a3d3-6ffcc3603bb5 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:18.345467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7e2b3-577c-435e-a3d3-6ffcc3603bb5 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:18.350204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca780a-b024-467e-b5e0-5b978e9d3ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:18.350860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca780a-b024-467e-b5e0-5b978e9d3ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:18.351146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca780a-b024-467e-b5e0-5b978e9d3ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:18.351425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca780a-b024-467e-b5e0-5b978e9d3ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:18.352146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca780a-b024-467e-b5e0-5b978e9d3ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:18.353830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca780a-b024-467e-b5e0-5b978e9d3ba3 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:18.364582 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1252/2473] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:18.368899 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8af542-612a-460a-95b6-f6fb6de3694b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:18.380658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8af542-612a-460a-95b6-f6fb6de3694b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:18.385476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8eb7bd-bc5d-472d-8583-feefa0c84f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:18.386893 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8eb7bd-bc5d-472d-8583-feefa0c84f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:18.387180 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8eb7bd-bc5d-472d-8583-feefa0c84f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:18.387638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8eb7bd-bc5d-472d-8583-feefa0c84f5a admin admin] PATH_INFO: `/v3/projects/b7196953c5444dd0a33757a30e0ae5c9/users/26da5516166547398c76108ab283d811/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:18.403739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8eb7bd-bc5d-472d-8583-feefa0c84f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7196953c5444dd0a33757a30e0ae5c9, user_id=26da5516166547398c76108ab283d811, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:18.406430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8eb7bd-bc5d-472d-8583-feefa0c84f5a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:18.421194 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1222/2474] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:18 2022] PUT /identity/v3/projects/b7196953c5444dd0a33757a30e0ae5c9/users/26da5516166547398c76108ab283d811/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:11:18.424957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca780a-b024-467e-b5e0-5b978e9d3ba3 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:18.435594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca780a-b024-467e-b5e0-5b978e9d3ba3 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:18.474534 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:18.504477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db39cf-5892-480b-a132-0b50c78c961e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:18.505300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db39cf-5892-480b-a132-0b50c78c961e admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:18.505591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db39cf-5892-480b-a132-0b50c78c961e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:18.505914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db39cf-5892-480b-a132-0b50c78c961e admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:18.506933 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db39cf-5892-480b-a132-0b50c78c961e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:18.508861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db39cf-5892-480b-a132-0b50c78c961e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:18.521732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1253/2475] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:18.524393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8eb7bd-bc5d-472d-8583-feefa0c84f5a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:18.535854 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8eb7bd-bc5d-472d-8583-feefa0c84f5a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:18.541512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf92ce-c0d5-470a-9538-e6b4c595b9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:18.542744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf92ce-c0d5-470a-9538-e6b4c595b9f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:18.543190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf92ce-c0d5-470a-9538-e6b4c595b9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:18.543533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf92ce-c0d5-470a-9538-e6b4c595b9f7 admin admin] PATH_INFO: `/v3/projects/b7196953c5444dd0a33757a30e0ae5c9/users/26da5516166547398c76108ab283d811/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:18.547588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf92ce-c0d5-470a-9538-e6b4c595b9f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7196953c5444dd0a33757a30e0ae5c9, user_id=26da5516166547398c76108ab283d811, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:18.551593 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf92ce-c0d5-470a-9538-e6b4c595b9f7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:18.564103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1223/2476] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:18 2022] PUT /identity/v3/projects/b7196953c5444dd0a33757a30e0ae5c9/users/26da5516166547398c76108ab283d811/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:11:18.576400 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316615f-2fd7-4f3d-9085-afa178ff8b44 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:18.576733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316615f-2fd7-4f3d-9085-afa178ff8b44 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:18.577045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316615f-2fd7-4f3d-9085-afa178ff8b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:18.621877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-2316615f-2fd7-4f3d-9085-afa178ff8b44 None None] MFA Rules not processed for user `26da5516166547398c76108ab283d811`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:18.623109 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-2316615f-2fd7-4f3d-9085-afa178ff8b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:18.694647 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1254/2477] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:18 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:18.706853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf92ce-c0d5-470a-9538-e6b4c595b9f7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:18.708769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5fbf92ce-c0d5-470a-9538-e6b4c595b9f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:18.766991 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf92ce-c0d5-470a-9538-e6b4c595b9f7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:18.770652 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf47180-0137-456b-ae16-1e1f4b3a2138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:18.771298 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf47180-0137-456b-ae16-1e1f4b3a2138 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:18.771586 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf47180-0137-456b-ae16-1e1f4b3a2138 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:18.771866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf47180-0137-456b-ae16-1e1f4b3a2138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:18.778422 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:18.782832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf47180-0137-456b-ae16-1e1f4b3a2138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:18.784553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf47180-0137-456b-ae16-1e1f4b3a2138 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:18.794660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1224/2478] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:18 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:18.867431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316615f-2fd7-4f3d-9085-afa178ff8b44 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:18.878196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316615f-2fd7-4f3d-9085-afa178ff8b44 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:18.881812 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbf055-b414-4aec-9324-bc4e2800c65d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:18.882448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbf055-b414-4aec-9324-bc4e2800c65d service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:18.882733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbf055-b414-4aec-9324-bc4e2800c65d service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:18.883012 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbf055-b414-4aec-9324-bc4e2800c65d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:18.940959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbf055-b414-4aec-9324-bc4e2800c65d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:18.942431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbf055-b414-4aec-9324-bc4e2800c65d service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:18.951789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1255/2479] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:18 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:22.650252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf47180-0137-456b-ae16-1e1f4b3a2138 service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:22.651843 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9cf47180-0137-456b-ae16-1e1f4b3a2138 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:22.729382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf47180-0137-456b-ae16-1e1f4b3a2138 service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:22.736107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac35321-6fd0-40f5-9954-035556764712 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:22.736761 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac35321-6fd0-40f5-9954-035556764712 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:22.737048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac35321-6fd0-40f5-9954-035556764712 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:22.737357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac35321-6fd0-40f5-9954-035556764712 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:22.739126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7ac35321-6fd0-40f5-9954-035556764712 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:22.751508 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac35321-6fd0-40f5-9954-035556764712 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:22.753182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac35321-6fd0-40f5-9954-035556764712 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:22.763702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1225/2480] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:26.043675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbf055-b414-4aec-9324-bc4e2800c65d service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:26.045129 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6bcbf055-b414-4aec-9324-bc4e2800c65d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:26.116710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:26.129430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbf055-b414-4aec-9324-bc4e2800c65d service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:26.134282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:26.135168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:26.135453 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:26.135725 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] PATH_INFO: `/v3/users/1c8d9f15a131430d92060bea3cd13c96` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:26.137538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8d9f15a131430d92060bea3cd13c96)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:26.147525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:26.215316 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8d9f15a131430d92060bea3cd13c96'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:26.216166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8d9f15a131430d92060bea3cd13c96'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:26.228133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8d9f15a131430d92060bea3cd13c96'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:26.240300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8d9f15a131430d92060bea3cd13c96'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:26.241008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8d9f15a131430d92060bea3cd13c96'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:26.254319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1256/2481] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:26 2022] DELETE /identity/v3/users/1c8d9f15a131430d92060bea3cd13c96 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:26.258880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1881df-b2d8-401c-b90b-f5332222314f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:26.259381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1881df-b2d8-401c-b90b-f5332222314f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:26.259514 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1881df-b2d8-401c-b90b-f5332222314f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:26.278889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1e1881df-b2d8-401c-b90b-f5332222314f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:26.280099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1e1881df-b2d8-401c-b90b-f5332222314f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:26.347044 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1226/2482] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:26.356671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:26.358290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:26.438223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6031b28-4d80-46f2-98ad-55b397f7e87e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:26.444701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6655516-fc59-4376-855c-210179ab2ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:26.445548 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6655516-fc59-4376-855c-210179ab2ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:26.456532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6655516-fc59-4376-855c-210179ab2ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:26.456858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6655516-fc59-4376-855c-210179ab2ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:26.472428 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6655516-fc59-4376-855c-210179ab2ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:26.473956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6655516-fc59-4376-855c-210179ab2ba5 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:26.493545 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1257/2483] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:26.801173 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1881df-b2d8-401c-b90b-f5332222314f None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:26.802990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1e1881df-b2d8-401c-b90b-f5332222314f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:26.816526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1881df-b2d8-401c-b90b-f5332222314f None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:26.821355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942a02e-10b1-4003-80e1-6da73f772e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:26.822441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942a02e-10b1-4003-80e1-6da73f772e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:26.822686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942a02e-10b1-4003-80e1-6da73f772e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:26.823044 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942a02e-10b1-4003-80e1-6da73f772e41 admin admin] PATH_INFO: `/v3/projects/2ffa5b55381f4f1b9676af05dbddff73` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:26.825049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942a02e-10b1-4003-80e1-6da73f772e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ffa5b55381f4f1b9676af05dbddff73)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:26.827660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942a02e-10b1-4003-80e1-6da73f772e41 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:26.879677 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0942a02e-10b1-4003-80e1-6da73f772e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ffa5b55381f4f1b9676af05dbddff73', 'request_id': 'req-0942a02e-10b1-4003-80e1-6da73f772e41'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:26.886374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0942a02e-10b1-4003-80e1-6da73f772e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ffa5b55381f4f1b9676af05dbddff73', 'request_id': 'req-0942a02e-10b1-4003-80e1-6da73f772e41'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:26.895123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0942a02e-10b1-4003-80e1-6da73f772e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ffa5b55381f4f1b9676af05dbddff73', 'request_id': 'req-0942a02e-10b1-4003-80e1-6da73f772e41'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:26.906907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0942a02e-10b1-4003-80e1-6da73f772e41 admin admin] The token cache is being invalidate because project 2ffa5b55381f4f1b9676af05dbddff73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:26.907192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0942a02e-10b1-4003-80e1-6da73f772e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:26.907826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0942a02e-10b1-4003-80e1-6da73f772e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:26.909374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1227/2484] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:26 2022] DELETE /identity/v3/projects/2ffa5b55381f4f1b9676af05dbddff73 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:26.914168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66803352-84e4-437b-8984-56bcd5b047df None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:26.914452 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66803352-84e4-437b-8984-56bcd5b047df None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:26.914742 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66803352-84e4-437b-8984-56bcd5b047df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:26.932470 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-66803352-84e4-437b-8984-56bcd5b047df None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:26.934491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-66803352-84e4-437b-8984-56bcd5b047df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:26.992983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1258/2485] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:27.002487 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2f75b-a02e-4d97-a1c9-f803edd750bc None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.002782 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2f75b-a02e-4d97-a1c9-f803edd750bc None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.003061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2f75b-a02e-4d97-a1c9-f803edd750bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.009473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:27.022976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-abb2f75b-a02e-4d97-a1c9-f803edd750bc None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:27.024187 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-abb2f75b-a02e-4d97-a1c9-f803edd750bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:27.032124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1228/2486] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:27.036249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66803352-84e4-437b-8984-56bcd5b047df None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:27.046455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66803352-84e4-437b-8984-56bcd5b047df None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:27.051094 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22dfb12-ea05-456d-9582-fc74dff79e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:27.051729 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22dfb12-ea05-456d-9582-fc74dff79e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.052013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22dfb12-ea05-456d-9582-fc74dff79e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.052288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22dfb12-ea05-456d-9582-fc74dff79e92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.053037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22dfb12-ea05-456d-9582-fc74dff79e92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:27.053380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22dfb12-ea05-456d-9582-fc74dff79e92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:27.054975 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22dfb12-ea05-456d-9582-fc74dff79e92 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:27.067778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1259/2487] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:27.071823 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2f75b-a02e-4d97-a1c9-f803edd750bc None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:27.081028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2f75b-a02e-4d97-a1c9-f803edd750bc None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:27.085688 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc130e15-0577-480d-bb9f-809a562d730e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:27.086333 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc130e15-0577-480d-bb9f-809a562d730e admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.086621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc130e15-0577-480d-bb9f-809a562d730e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.086901 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc130e15-0577-480d-bb9f-809a562d730e admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.087590 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc130e15-0577-480d-bb9f-809a562d730e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:27.089759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc130e15-0577-480d-bb9f-809a562d730e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:27.115175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1229/2488] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:27 2022] POST /identity/v3/projects => generated 357 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:27.119255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22dfb12-ea05-456d-9582-fc74dff79e92 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:27.129576 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22dfb12-ea05-456d-9582-fc74dff79e92 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:27.133049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:27.194573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9926b-927d-4643-8908-c9e89d394628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:27.195273 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9926b-927d-4643-8908-c9e89d394628 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.195560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9926b-927d-4643-8908-c9e89d394628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.195839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9926b-927d-4643-8908-c9e89d394628 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.196533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9926b-927d-4643-8908-c9e89d394628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:27.199054 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9926b-927d-4643-8908-c9e89d394628 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:27.218914 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1260/2489] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:27 2022] POST /identity/v3/users => generated 279 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:27.223232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc130e15-0577-480d-bb9f-809a562d730e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:27.233624 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc130e15-0577-480d-bb9f-809a562d730e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:27.238158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259e78d-5af4-4c74-8a4d-804eca96bfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:27.238803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259e78d-5af4-4c74-8a4d-804eca96bfb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.239082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259e78d-5af4-4c74-8a4d-804eca96bfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.239353 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259e78d-5af4-4c74-8a4d-804eca96bfb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.240062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3259e78d-5af4-4c74-8a4d-804eca96bfb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:27.241660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3259e78d-5af4-4c74-8a4d-804eca96bfb2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:27.252179 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1230/2490] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:27.255941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9926b-927d-4643-8908-c9e89d394628 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:27.266373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9926b-927d-4643-8908-c9e89d394628 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:27.270995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d760785-8c01-4232-ba49-abde4bdddc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:27.271649 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d760785-8c01-4232-ba49-abde4bdddc2b admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.271939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d760785-8c01-4232-ba49-abde4bdddc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.272220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d760785-8c01-4232-ba49-abde4bdddc2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.272945 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d760785-8c01-4232-ba49-abde4bdddc2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:27.274567 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d760785-8c01-4232-ba49-abde4bdddc2b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:27.284605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1261/2491] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:27.288660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259e78d-5af4-4c74-8a4d-804eca96bfb2 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:27.299000 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259e78d-5af4-4c74-8a4d-804eca96bfb2 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:27.303629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391a0ab-52e6-4685-9092-23997c67bb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:27.304486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391a0ab-52e6-4685-9092-23997c67bb20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.304765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391a0ab-52e6-4685-9092-23997c67bb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.305061 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391a0ab-52e6-4685-9092-23997c67bb20 admin admin] PATH_INFO: `/v3/projects/3614d0717bc24f6c9002397e09d5a497/users/61187cefa09d48bd981bfaebecba7493/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.320633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391a0ab-52e6-4685-9092-23997c67bb20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3614d0717bc24f6c9002397e09d5a497, user_id=61187cefa09d48bd981bfaebecba7493, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:27.323434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391a0ab-52e6-4685-9092-23997c67bb20 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:27.336167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1231/2492] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:27 2022] PUT /identity/v3/projects/3614d0717bc24f6c9002397e09d5a497/users/61187cefa09d48bd981bfaebecba7493/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:11:27.347203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda04a4a-300c-4007-a872-fe2f860dc4ec None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.347491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda04a4a-300c-4007-a872-fe2f860dc4ec None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.347792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda04a4a-300c-4007-a872-fe2f860dc4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.374955 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-fda04a4a-300c-4007-a872-fe2f860dc4ec None None] MFA Rules not processed for user `61187cefa09d48bd981bfaebecba7493`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:27.376174 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-fda04a4a-300c-4007-a872-fe2f860dc4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:27.439720 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1262/2493] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:27 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:27.444058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391a0ab-52e6-4685-9092-23997c67bb20 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:27.507656 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391a0ab-52e6-4685-9092-23997c67bb20 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:27.512550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac03b6-d777-4302-b22f-74bea2e51617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:27.513247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac03b6-d777-4302-b22f-74bea2e51617 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.513538 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac03b6-d777-4302-b22f-74bea2e51617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.513820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac03b6-d777-4302-b22f-74bea2e51617 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.514519 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac03b6-d777-4302-b22f-74bea2e51617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:27.516426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac03b6-d777-4302-b22f-74bea2e51617 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:27.543784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1232/2494] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:27 2022] POST /identity/v3/projects => generated 357 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:27.548380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda04a4a-300c-4007-a872-fe2f860dc4ec None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:27.559527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda04a4a-300c-4007-a872-fe2f860dc4ec None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:27.619349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e25271-fc86-462c-b6e6-788c0849e0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:27.620030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e25271-fc86-462c-b6e6-788c0849e0b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.620370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e25271-fc86-462c-b6e6-788c0849e0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.621247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e25271-fc86-462c-b6e6-788c0849e0b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.621969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e25271-fc86-462c-b6e6-788c0849e0b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:27.623907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e25271-fc86-462c-b6e6-788c0849e0b2 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:27.645076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1263/2495] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:27 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:27.648978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac03b6-d777-4302-b22f-74bea2e51617 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:27.659470 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac03b6-d777-4302-b22f-74bea2e51617 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:27.664130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275373c-5d56-4f94-a84a-84feb65e9374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:27.664791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275373c-5d56-4f94-a84a-84feb65e9374 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.665078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275373c-5d56-4f94-a84a-84feb65e9374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.665386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275373c-5d56-4f94-a84a-84feb65e9374 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.666140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275373c-5d56-4f94-a84a-84feb65e9374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:27.668494 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275373c-5d56-4f94-a84a-84feb65e9374 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:27.679454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1233/2496] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:27.684310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e25271-fc86-462c-b6e6-788c0849e0b2 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:27.690709 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:27.694550 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e25271-fc86-462c-b6e6-788c0849e0b2 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:27.700325 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c79107-f669-44a7-bc74-cfcd9985130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:27.701266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c79107-f669-44a7-bc74-cfcd9985130c admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.701555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c79107-f669-44a7-bc74-cfcd9985130c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.701833 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c79107-f669-44a7-bc74-cfcd9985130c admin admin] PATH_INFO: `/v3/projects/a9b13f26bd10455eb9259a18ae2d9ea5/users/36b4c1b4856d40c5ba42bd78d90b7c40/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.716638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c79107-f669-44a7-bc74-cfcd9985130c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b13f26bd10455eb9259a18ae2d9ea5, user_id=36b4c1b4856d40c5ba42bd78d90b7c40, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:27.719339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c79107-f669-44a7-bc74-cfcd9985130c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:27.733153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1264/2497] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:27 2022] PUT /identity/v3/projects/a9b13f26bd10455eb9259a18ae2d9ea5/users/36b4c1b4856d40c5ba42bd78d90b7c40/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:27.737073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275373c-5d56-4f94-a84a-84feb65e9374 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:27.747657 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275373c-5d56-4f94-a84a-84feb65e9374 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:27.803610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79ce35-5231-4dad-90a2-9cc0505f27a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:27.804274 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79ce35-5231-4dad-90a2-9cc0505f27a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.804556 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79ce35-5231-4dad-90a2-9cc0505f27a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.804834 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79ce35-5231-4dad-90a2-9cc0505f27a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.805585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79ce35-5231-4dad-90a2-9cc0505f27a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:27.807163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79ce35-5231-4dad-90a2-9cc0505f27a7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:27.817652 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1234/2498] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:27.821707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c79107-f669-44a7-bc74-cfcd9985130c admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:27.832211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c79107-f669-44a7-bc74-cfcd9985130c admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:27.836831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77387e65-6bf1-4830-b3ea-0054ac613025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:27.837743 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77387e65-6bf1-4830-b3ea-0054ac613025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.838031 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77387e65-6bf1-4830-b3ea-0054ac613025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.838310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77387e65-6bf1-4830-b3ea-0054ac613025 admin admin] PATH_INFO: `/v3/projects/a9b13f26bd10455eb9259a18ae2d9ea5/users/36b4c1b4856d40c5ba42bd78d90b7c40/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.840416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77387e65-6bf1-4830-b3ea-0054ac613025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b13f26bd10455eb9259a18ae2d9ea5, user_id=36b4c1b4856d40c5ba42bd78d90b7c40, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:27.843107 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77387e65-6bf1-4830-b3ea-0054ac613025 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:27.853408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1265/2499] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:27 2022] PUT /identity/v3/projects/a9b13f26bd10455eb9259a18ae2d9ea5/users/36b4c1b4856d40c5ba42bd78d90b7c40/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:27.864067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c28b6-0981-4c96-8b43-0e2e2da49a31 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:27.864364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c28b6-0981-4c96-8b43-0e2e2da49a31 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:27.864650 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c28b6-0981-4c96-8b43-0e2e2da49a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:27.883359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:27.895109 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ec9c28b6-0981-4c96-8b43-0e2e2da49a31 None None] MFA Rules not processed for user `36b4c1b4856d40c5ba42bd78d90b7c40`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:27.896429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ec9c28b6-0981-4c96-8b43-0e2e2da49a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:27.969957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1235/2500] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:27 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:27.982101 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77387e65-6bf1-4830-b3ea-0054ac613025 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:27.983802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-77387e65-6bf1-4830-b3ea-0054ac613025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:28.043247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77387e65-6bf1-4830-b3ea-0054ac613025 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:28.046974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c731f-ab70-41fc-853a-2801c8c06029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:28.047617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c731f-ab70-41fc-853a-2801c8c06029 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:28.047905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c731f-ab70-41fc-853a-2801c8c06029 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:28.048187 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c731f-ab70-41fc-853a-2801c8c06029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:28.057396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c731f-ab70-41fc-853a-2801c8c06029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:28.059007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c731f-ab70-41fc-853a-2801c8c06029 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:28.068276 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1266/2501] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:27 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:28.139933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c28b6-0981-4c96-8b43-0e2e2da49a31 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:28.149446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c28b6-0981-4c96-8b43-0e2e2da49a31 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:28.153714 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fceb27-1da4-446d-88ac-0986c40ced35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:28.154351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fceb27-1da4-446d-88ac-0986c40ced35 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:28.154636 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fceb27-1da4-446d-88ac-0986c40ced35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:28.154912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fceb27-1da4-446d-88ac-0986c40ced35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:28.210731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fceb27-1da4-446d-88ac-0986c40ced35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:28.212308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fceb27-1da4-446d-88ac-0986c40ced35 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:28.222423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1236/2502] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:28 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:35.801401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c731f-ab70-41fc-853a-2801c8c06029 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:35.802847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f36c731f-ab70-41fc-853a-2801c8c06029 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:35.873668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c731f-ab70-41fc-853a-2801c8c06029 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:35.878536 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:35.879435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:35.879719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:35.879996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] PATH_INFO: `/v3/users/5662bdf849904077a21d3b0e34afe5a5` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:35.881215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5662bdf849904077a21d3b0e34afe5a5)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:35.883689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:35.934318 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5662bdf849904077a21d3b0e34afe5a5'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:35.934980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5662bdf849904077a21d3b0e34afe5a5'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:35.941839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5662bdf849904077a21d3b0e34afe5a5'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:35.949432 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:35.955502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5662bdf849904077a21d3b0e34afe5a5'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:35.956681 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5662bdf849904077a21d3b0e34afe5a5'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:35.970044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1267/2503] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:35 2022] DELETE /identity/v3/users/5662bdf849904077a21d3b0e34afe5a5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:35.974799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00d2ab-5e8c-4fb8-8eca-6fcefad70e8c None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:35.975275 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00d2ab-5e8c-4fb8-8eca-6fcefad70e8c None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:35.975381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00d2ab-5e8c-4fb8-8eca-6fcefad70e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:35.992005 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2e00d2ab-5e8c-4fb8-8eca-6fcefad70e8c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:35.993231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2e00d2ab-5e8c-4fb8-8eca-6fcefad70e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:36.053141 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1237/2504] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:36.062678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:36.064069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:36.121740 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ff817-c606-4975-8a86-ae5acbb84fd1 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:36.125493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1a1ff-8886-4e26-a2b3-a3fb6c107760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:36.126081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1a1ff-8886-4e26-a2b3-a3fb6c107760 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:36.126371 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1a1ff-8886-4e26-a2b3-a3fb6c107760 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:36.126649 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1a1ff-8886-4e26-a2b3-a3fb6c107760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:36.136296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1a1ff-8886-4e26-a2b3-a3fb6c107760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:36.137763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1a1ff-8886-4e26-a2b3-a3fb6c107760 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:36.147171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1268/2505] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:36.425895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00d2ab-5e8c-4fb8-8eca-6fcefad70e8c None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:36.427356 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2e00d2ab-5e8c-4fb8-8eca-6fcefad70e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:36.440475 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00d2ab-5e8c-4fb8-8eca-6fcefad70e8c None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:36.445532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:36.453794 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:36.454082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:36.454359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] PATH_INFO: `/v3/projects/eda8f759205b43e4859e6251411eee7d` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:36.455605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eda8f759205b43e4859e6251411eee7d)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:36.457557 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:36.490485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:36.506485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eda8f759205b43e4859e6251411eee7d', 'request_id': 'req-f1664359-553f-4c7c-bcc0-cd4aba935d69'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:36.512034 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eda8f759205b43e4859e6251411eee7d', 'request_id': 'req-f1664359-553f-4c7c-bcc0-cd4aba935d69'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:36.519302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eda8f759205b43e4859e6251411eee7d', 'request_id': 'req-f1664359-553f-4c7c-bcc0-cd4aba935d69'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:36.530740 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] The token cache is being invalidate because project eda8f759205b43e4859e6251411eee7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:36.531025 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:36.531691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:36.533288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1238/2506] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:36 2022] DELETE /identity/v3/projects/eda8f759205b43e4859e6251411eee7d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:36.537206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1a1ff-8886-4e26-a2b3-a3fb6c107760 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:36.548955 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1a1ff-8886-4e26-a2b3-a3fb6c107760 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:36.605742 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb3889-4a28-4761-b90d-b2331191ba36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:36.606619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb3889-4a28-4761-b90d-b2331191ba36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:36.606907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb3889-4a28-4761-b90d-b2331191ba36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:36.607189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb3889-4a28-4761-b90d-b2331191ba36 admin admin] PATH_INFO: `/v3/users/26da5516166547398c76108ab283d811` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:36.608376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb3889-4a28-4761-b90d-b2331191ba36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26da5516166547398c76108ab283d811)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:36.610330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb3889-4a28-4761-b90d-b2331191ba36 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:36.656785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-39fb3889-4a28-4761-b90d-b2331191ba36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26da5516166547398c76108ab283d811'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:36.657475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-39fb3889-4a28-4761-b90d-b2331191ba36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26da5516166547398c76108ab283d811'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:36.664505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-39fb3889-4a28-4761-b90d-b2331191ba36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26da5516166547398c76108ab283d811'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:36.676740 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-39fb3889-4a28-4761-b90d-b2331191ba36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26da5516166547398c76108ab283d811'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:36.677464 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-39fb3889-4a28-4761-b90d-b2331191ba36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26da5516166547398c76108ab283d811'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:36.693464 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1269/2507] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:36 2022] DELETE /identity/v3/users/26da5516166547398c76108ab283d811 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:36.974208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:36.976332 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:36.996697 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1664359-553f-4c7c-bcc0-cd4aba935d69 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:37.068751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992558b-b070-4025-81d4-f9dc44bff701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:37.069785 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992558b-b070-4025-81d4-f9dc44bff701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.070083 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992558b-b070-4025-81d4-f9dc44bff701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.070320 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992558b-b070-4025-81d4-f9dc44bff701 admin admin] PATH_INFO: `/v3/projects/b7196953c5444dd0a33757a30e0ae5c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.071499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992558b-b070-4025-81d4-f9dc44bff701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7196953c5444dd0a33757a30e0ae5c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:37.073883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992558b-b070-4025-81d4-f9dc44bff701 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:37.125887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e992558b-b070-4025-81d4-f9dc44bff701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7196953c5444dd0a33757a30e0ae5c9', 'request_id': 'req-e992558b-b070-4025-81d4-f9dc44bff701'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:37.131573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e992558b-b070-4025-81d4-f9dc44bff701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7196953c5444dd0a33757a30e0ae5c9', 'request_id': 'req-e992558b-b070-4025-81d4-f9dc44bff701'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:37.143694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e992558b-b070-4025-81d4-f9dc44bff701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7196953c5444dd0a33757a30e0ae5c9', 'request_id': 'req-e992558b-b070-4025-81d4-f9dc44bff701'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:37.158198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e992558b-b070-4025-81d4-f9dc44bff701 admin admin] The token cache is being invalidate because project b7196953c5444dd0a33757a30e0ae5c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:37.158457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e992558b-b070-4025-81d4-f9dc44bff701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:37.159113 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e992558b-b070-4025-81d4-f9dc44bff701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:37.160660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1239/2508] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:36 2022] DELETE /identity/v3/projects/b7196953c5444dd0a33757a30e0ae5c9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:37.168140 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594a141-01ee-4e36-b27a-29d74f70d163 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.168660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594a141-01ee-4e36-b27a-29d74f70d163 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.168806 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594a141-01ee-4e36-b27a-29d74f70d163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.176552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:37.188873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1594a141-01ee-4e36-b27a-29d74f70d163 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:37.190032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1594a141-01ee-4e36-b27a-29d74f70d163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:37.249285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1270/2509] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:11:37.257247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47001687-e909-4b30-a12e-b91e844f720f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.257537 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47001687-e909-4b30-a12e-b91e844f720f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.257836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47001687-e909-4b30-a12e-b91e844f720f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.278928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-47001687-e909-4b30-a12e-b91e844f720f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:37.280123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-47001687-e909-4b30-a12e-b91e844f720f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:37.288313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1240/2510] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:37.292456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594a141-01ee-4e36-b27a-29d74f70d163 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:37.302189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594a141-01ee-4e36-b27a-29d74f70d163 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:37.307047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2400c3-0ec9-4a63-ba0e-1d1b9b4bf198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:37.308384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2400c3-0ec9-4a63-ba0e-1d1b9b4bf198 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.308801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2400c3-0ec9-4a63-ba0e-1d1b9b4bf198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.309190 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2400c3-0ec9-4a63-ba0e-1d1b9b4bf198 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.309993 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2400c3-0ec9-4a63-ba0e-1d1b9b4bf198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:37.310373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2400c3-0ec9-4a63-ba0e-1d1b9b4bf198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:37.312123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2400c3-0ec9-4a63-ba0e-1d1b9b4bf198 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:37.324735 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1271/2511] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:37.329382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47001687-e909-4b30-a12e-b91e844f720f None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:37.339797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47001687-e909-4b30-a12e-b91e844f720f None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:37.344669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c25b19-1db8-468a-85f8-29d32436700b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:37.345645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c25b19-1db8-468a-85f8-29d32436700b admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.346088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c25b19-1db8-468a-85f8-29d32436700b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.346370 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c25b19-1db8-468a-85f8-29d32436700b admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.347142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c25b19-1db8-468a-85f8-29d32436700b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:37.349442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c25b19-1db8-468a-85f8-29d32436700b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:37.377139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1241/2512] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:37 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:37.381338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2400c3-0ec9-4a63-ba0e-1d1b9b4bf198 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:37.391429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2400c3-0ec9-4a63-ba0e-1d1b9b4bf198 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:37.447496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6276f0-5946-4027-ba20-3614448f66f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:37.448105 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6276f0-5946-4027-ba20-3614448f66f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.448396 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6276f0-5946-4027-ba20-3614448f66f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.448680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6276f0-5946-4027-ba20-3614448f66f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.449412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6276f0-5946-4027-ba20-3614448f66f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:37.451342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6276f0-5946-4027-ba20-3614448f66f5 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:37.472446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1272/2513] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:37 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:37.475986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c25b19-1db8-468a-85f8-29d32436700b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:37.489393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c25b19-1db8-468a-85f8-29d32436700b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:37.494871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dfb52-9a0c-48bf-b781-0eab7d24a556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:37.496190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dfb52-9a0c-48bf-b781-0eab7d24a556 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.496549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dfb52-9a0c-48bf-b781-0eab7d24a556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.496912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dfb52-9a0c-48bf-b781-0eab7d24a556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.497666 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589dfb52-9a0c-48bf-b781-0eab7d24a556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:37.499276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589dfb52-9a0c-48bf-b781-0eab7d24a556 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:37.511376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1242/2514] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:37.516359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6276f0-5946-4027-ba20-3614448f66f5 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:37.528366 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6276f0-5946-4027-ba20-3614448f66f5 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:37.533005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53665e7b-7ffa-4d8a-992a-ead29609c4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:37.533714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53665e7b-7ffa-4d8a-992a-ead29609c4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.533997 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53665e7b-7ffa-4d8a-992a-ead29609c4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.534270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53665e7b-7ffa-4d8a-992a-ead29609c4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.534986 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53665e7b-7ffa-4d8a-992a-ead29609c4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:37.536568 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53665e7b-7ffa-4d8a-992a-ead29609c4e9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:37.547403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1273/2515] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:37.551585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dfb52-9a0c-48bf-b781-0eab7d24a556 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:37.562087 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dfb52-9a0c-48bf-b781-0eab7d24a556 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:37.566642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e22547-e774-462d-875c-6ff58f8cc536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:37.567505 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e22547-e774-462d-875c-6ff58f8cc536 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.567784 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e22547-e774-462d-875c-6ff58f8cc536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.568058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e22547-e774-462d-875c-6ff58f8cc536 admin admin] PATH_INFO: `/v3/projects/4fbb8d83807f4161984729cde0d91e3a/users/ece4862c79ca4e9e93df61446780307f/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.570263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:37.585843 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e22547-e774-462d-875c-6ff58f8cc536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fbb8d83807f4161984729cde0d91e3a, user_id=ece4862c79ca4e9e93df61446780307f, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:37.588668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e22547-e774-462d-875c-6ff58f8cc536 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:37.601670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1243/2516] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:37 2022] PUT /identity/v3/projects/4fbb8d83807f4161984729cde0d91e3a/users/ece4862c79ca4e9e93df61446780307f/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:37.613429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854cebc-64b2-4999-b442-7629004fb18d None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.613713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854cebc-64b2-4999-b442-7629004fb18d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.613990 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854cebc-64b2-4999-b442-7629004fb18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.641443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-2854cebc-64b2-4999-b442-7629004fb18d None None] MFA Rules not processed for user `ece4862c79ca4e9e93df61446780307f`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:37.642654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-2854cebc-64b2-4999-b442-7629004fb18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:37.702245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:37.706165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1274/2517] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:37.710485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e22547-e774-462d-875c-6ff58f8cc536 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:37.773453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e22547-e774-462d-875c-6ff58f8cc536 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:37.780642 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77115b5-a6d2-44d9-829e-a59f613feef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:37.781330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77115b5-a6d2-44d9-829e-a59f613feef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.781618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77115b5-a6d2-44d9-829e-a59f613feef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.781896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77115b5-a6d2-44d9-829e-a59f613feef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.782574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77115b5-a6d2-44d9-829e-a59f613feef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:37.784465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77115b5-a6d2-44d9-829e-a59f613feef1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:37.809837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1244/2518] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:37 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:37.813892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854cebc-64b2-4999-b442-7629004fb18d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:37.825312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854cebc-64b2-4999-b442-7629004fb18d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:37.880662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0375f-e431-44a7-9c57-c0cb61aedde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:37.881344 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0375f-e431-44a7-9c57-c0cb61aedde6 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.881633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0375f-e431-44a7-9c57-c0cb61aedde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.881911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0375f-e431-44a7-9c57-c0cb61aedde6 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.882599 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0375f-e431-44a7-9c57-c0cb61aedde6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:37.884762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0375f-e431-44a7-9c57-c0cb61aedde6 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:37.905866 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1275/2519] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:37 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:37.909584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77115b5-a6d2-44d9-829e-a59f613feef1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:37.920305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77115b5-a6d2-44d9-829e-a59f613feef1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:37.924920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee488d-871f-4027-9b5f-43c23569abc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:37.925605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee488d-871f-4027-9b5f-43c23569abc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.925887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee488d-871f-4027-9b5f-43c23569abc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.926168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee488d-871f-4027-9b5f-43c23569abc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.926883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee488d-871f-4027-9b5f-43c23569abc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:37.928485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee488d-871f-4027-9b5f-43c23569abc0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:37.939486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1245/2520] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:37.943413 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0375f-e431-44a7-9c57-c0cb61aedde6 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:37.953978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0375f-e431-44a7-9c57-c0cb61aedde6 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:37.958816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9a2bc-fcd7-4c22-a33c-76094966996c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:37.959686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9a2bc-fcd7-4c22-a33c-76094966996c admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:37.959971 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9a2bc-fcd7-4c22-a33c-76094966996c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:37.960246 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9a2bc-fcd7-4c22-a33c-76094966996c admin admin] PATH_INFO: `/v3/projects/43d52e7dbd154bf6a24e8b1a360f5909/users/51251a8213e2403b89e34876ad6de2ec/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:37.976137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9a2bc-fcd7-4c22-a33c-76094966996c admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d52e7dbd154bf6a24e8b1a360f5909, user_id=51251a8213e2403b89e34876ad6de2ec, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:37.978830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9a2bc-fcd7-4c22-a33c-76094966996c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:37.991849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1276/2521] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:37 2022] PUT /identity/v3/projects/43d52e7dbd154bf6a24e8b1a360f5909/users/51251a8213e2403b89e34876ad6de2ec/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:37.996152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee488d-871f-4027-9b5f-43c23569abc0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:38.006568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee488d-871f-4027-9b5f-43c23569abc0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:38.062932 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31651cd5-028e-440a-bf96-bf71c9c1ba2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:38.063601 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31651cd5-028e-440a-bf96-bf71c9c1ba2b admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:38.063887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31651cd5-028e-440a-bf96-bf71c9c1ba2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:38.064171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31651cd5-028e-440a-bf96-bf71c9c1ba2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:38.064888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31651cd5-028e-440a-bf96-bf71c9c1ba2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:38.066523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31651cd5-028e-440a-bf96-bf71c9c1ba2b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:38.076777 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1246/2522] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:38.080857 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9a2bc-fcd7-4c22-a33c-76094966996c admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:38.091285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9a2bc-fcd7-4c22-a33c-76094966996c admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:38.096166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf655327-f898-462c-a48e-f7b6fbba0d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:38.097039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf655327-f898-462c-a48e-f7b6fbba0d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:38.097360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf655327-f898-462c-a48e-f7b6fbba0d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:38.097640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf655327-f898-462c-a48e-f7b6fbba0d26 admin admin] PATH_INFO: `/v3/projects/43d52e7dbd154bf6a24e8b1a360f5909/users/51251a8213e2403b89e34876ad6de2ec/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:38.099581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf655327-f898-462c-a48e-f7b6fbba0d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d52e7dbd154bf6a24e8b1a360f5909, user_id=51251a8213e2403b89e34876ad6de2ec, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:38.102244 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf655327-f898-462c-a48e-f7b6fbba0d26 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:38.112956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1277/2523] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:38 2022] PUT /identity/v3/projects/43d52e7dbd154bf6a24e8b1a360f5909/users/51251a8213e2403b89e34876ad6de2ec/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:38.123378 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e70cd-b4a4-41d6-a430-e6cf200b93e5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:38.123735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e70cd-b4a4-41d6-a430-e6cf200b93e5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:38.124119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e70cd-b4a4-41d6-a430-e6cf200b93e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:38.149604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ff6e70cd-b4a4-41d6-a430-e6cf200b93e5 None None] MFA Rules not processed for user `51251a8213e2403b89e34876ad6de2ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:38.150792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ff6e70cd-b4a4-41d6-a430-e6cf200b93e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:38.193377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:38.222880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1247/2524] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:38.233937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf655327-f898-462c-a48e-f7b6fbba0d26 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:38.235701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-bf655327-f898-462c-a48e-f7b6fbba0d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:38.293624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf655327-f898-462c-a48e-f7b6fbba0d26 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:38.297638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e60174-f69c-4481-b84e-42307a489c9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:38.298288 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e60174-f69c-4481-b84e-42307a489c9a service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:38.298564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e60174-f69c-4481-b84e-42307a489c9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:38.298837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e60174-f69c-4481-b84e-42307a489c9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:38.356447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e60174-f69c-4481-b84e-42307a489c9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:38.358685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e60174-f69c-4481-b84e-42307a489c9a service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:38.369601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1278/2525] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:41.147253 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e70cd-b4a4-41d6-a430-e6cf200b93e5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:41.210902 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e70cd-b4a4-41d6-a430-e6cf200b93e5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:41.216343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:41.217244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:41.217526 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:41.217799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] PATH_INFO: `/v3/users/ece4862c79ca4e9e93df61446780307f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:41.218986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ece4862c79ca4e9e93df61446780307f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:41.221158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:41.271767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ece4862c79ca4e9e93df61446780307f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:41.271767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ece4862c79ca4e9e93df61446780307f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:41.279296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ece4862c79ca4e9e93df61446780307f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:41.280093 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ece4862c79ca4e9e93df61446780307f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:41.295165 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ece4862c79ca4e9e93df61446780307f'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:41.309168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1248/2526] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:41 2022] DELETE /identity/v3/users/ece4862c79ca4e9e93df61446780307f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:41.313564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af964ee2-9b8f-42fd-9ed0-46faae202167 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:41.314055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af964ee2-9b8f-42fd-9ed0-46faae202167 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:41.314172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af964ee2-9b8f-42fd-9ed0-46faae202167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:41.326455 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:41.333106 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-af964ee2-9b8f-42fd-9ed0-46faae202167 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:41.334340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-af964ee2-9b8f-42fd-9ed0-46faae202167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:41.394426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1279/2527] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:41.404033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:41.405462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:41.466775 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4b765-f25f-40af-80c2-45688ec38411 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:41.470761 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef0916-437c-466e-bf2f-27e4ca966fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:41.471416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef0916-437c-466e-bf2f-27e4ca966fea service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:41.471701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef0916-437c-466e-bf2f-27e4ca966fea service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:41.472099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef0916-437c-466e-bf2f-27e4ca966fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:41.482107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef0916-437c-466e-bf2f-27e4ca966fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:41.483550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef0916-437c-466e-bf2f-27e4ca966fea service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:41.488665 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:41.494575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1249/2528] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:41.794971 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af964ee2-9b8f-42fd-9ed0-46faae202167 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:41.796476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-af964ee2-9b8f-42fd-9ed0-46faae202167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:41.810417 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af964ee2-9b8f-42fd-9ed0-46faae202167 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:41.815162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:41.816024 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:41.816311 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:41.816591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] PATH_INFO: `/v3/projects/4fbb8d83807f4161984729cde0d91e3a` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:41.817903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fbb8d83807f4161984729cde0d91e3a)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:41.826075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:41.873523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fbb8d83807f4161984729cde0d91e3a', 'request_id': 'req-239a4074-1b45-4037-9614-a7ef3aea8109'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:41.879465 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fbb8d83807f4161984729cde0d91e3a', 'request_id': 'req-239a4074-1b45-4037-9614-a7ef3aea8109'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:41.888901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fbb8d83807f4161984729cde0d91e3a', 'request_id': 'req-239a4074-1b45-4037-9614-a7ef3aea8109'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:41.904419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] The token cache is being invalidate because project 4fbb8d83807f4161984729cde0d91e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:41.904802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:41.905466 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:41.907176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1280/2529] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:41 2022] DELETE /identity/v3/projects/4fbb8d83807f4161984729cde0d91e3a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:41.911149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef0916-437c-466e-bf2f-27e4ca966fea service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:41.920922 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef0916-437c-466e-bf2f-27e4ca966fea service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:41.982323 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d11ecd-f1d3-403e-84aa-f2cf53bcc837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:41.983240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d11ecd-f1d3-403e-84aa-f2cf53bcc837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:41.983525 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d11ecd-f1d3-403e-84aa-f2cf53bcc837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:41.983804 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d11ecd-f1d3-403e-84aa-f2cf53bcc837 admin admin] PATH_INFO: `/v3/users/51251a8213e2403b89e34876ad6de2ec` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:41.984984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d11ecd-f1d3-403e-84aa-f2cf53bcc837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51251a8213e2403b89e34876ad6de2ec)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:41.986992 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d11ecd-f1d3-403e-84aa-f2cf53bcc837 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:42.034170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e9d11ecd-f1d3-403e-84aa-f2cf53bcc837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51251a8213e2403b89e34876ad6de2ec'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:42.034846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e9d11ecd-f1d3-403e-84aa-f2cf53bcc837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51251a8213e2403b89e34876ad6de2ec'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:42.043330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e9d11ecd-f1d3-403e-84aa-f2cf53bcc837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51251a8213e2403b89e34876ad6de2ec'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:42.044032 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e9d11ecd-f1d3-403e-84aa-f2cf53bcc837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51251a8213e2403b89e34876ad6de2ec'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:42.057753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-e9d11ecd-f1d3-403e-84aa-f2cf53bcc837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51251a8213e2403b89e34876ad6de2ec'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:42.070680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1250/2530] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:41 2022] DELETE /identity/v3/users/51251a8213e2403b89e34876ad6de2ec => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:42.514862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:42.517168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:42.531477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a4074-1b45-4037-9614-a7ef3aea8109 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:42.638898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa261039-6218-4a8e-b14b-23a3c37ae5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:42.640042 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa261039-6218-4a8e-b14b-23a3c37ae5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:42.640393 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa261039-6218-4a8e-b14b-23a3c37ae5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:42.640816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa261039-6218-4a8e-b14b-23a3c37ae5f9 admin admin] PATH_INFO: `/v3/projects/43d52e7dbd154bf6a24e8b1a360f5909` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:42.642087 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa261039-6218-4a8e-b14b-23a3c37ae5f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43d52e7dbd154bf6a24e8b1a360f5909)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:42.643960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa261039-6218-4a8e-b14b-23a3c37ae5f9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:42.683506 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:42.700491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fa261039-6218-4a8e-b14b-23a3c37ae5f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43d52e7dbd154bf6a24e8b1a360f5909', 'request_id': 'req-fa261039-6218-4a8e-b14b-23a3c37ae5f9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:42.707639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fa261039-6218-4a8e-b14b-23a3c37ae5f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43d52e7dbd154bf6a24e8b1a360f5909', 'request_id': 'req-fa261039-6218-4a8e-b14b-23a3c37ae5f9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:42.723071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fa261039-6218-4a8e-b14b-23a3c37ae5f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43d52e7dbd154bf6a24e8b1a360f5909', 'request_id': 'req-fa261039-6218-4a8e-b14b-23a3c37ae5f9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:42.736421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fa261039-6218-4a8e-b14b-23a3c37ae5f9 admin admin] The token cache is being invalidate because project 43d52e7dbd154bf6a24e8b1a360f5909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:42.736820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fa261039-6218-4a8e-b14b-23a3c37ae5f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:42.737412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fa261039-6218-4a8e-b14b-23a3c37ae5f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:42.739556 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1281/2531] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:42 2022] DELETE /identity/v3/projects/43d52e7dbd154bf6a24e8b1a360f5909 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:42.744169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc83ee-543b-4c7e-a862-8c3b27af0fb1 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:42.744658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc83ee-543b-4c7e-a862-8c3b27af0fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:42.744767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc83ee-543b-4c7e-a862-8c3b27af0fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:42.767834 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-2bcc83ee-543b-4c7e-a862-8c3b27af0fb1 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:42.769099 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2bcc83ee-543b-4c7e-a862-8c3b27af0fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:42.832879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1251/2532] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:42.841260 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a10d-4880-4460-9c63-a958a7054f4a None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:42.841543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a10d-4880-4460-9c63-a958a7054f4a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:42.841820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a10d-4880-4460-9c63-a958a7054f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:42.862158 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-da47a10d-4880-4460-9c63-a958a7054f4a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:42.863171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc83ee-543b-4c7e-a862-8c3b27af0fb1 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:42.863348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-da47a10d-4880-4460-9c63-a958a7054f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:42.864552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2bcc83ee-543b-4c7e-a862-8c3b27af0fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:42.871406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1282/2533] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:11:42.875422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47a10d-4880-4460-9c63-a958a7054f4a None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:42.884407 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47a10d-4880-4460-9c63-a958a7054f4a None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:42.891060 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:42.892038 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad515b-b744-45b2-a576-3980e7e4df36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:42.892705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad515b-b744-45b2-a576-3980e7e4df36 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:42.892986 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad515b-b744-45b2-a576-3980e7e4df36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:42.893338 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad515b-b744-45b2-a576-3980e7e4df36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:42.894070 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad515b-b744-45b2-a576-3980e7e4df36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:42.894368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad515b-b744-45b2-a576-3980e7e4df36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:42.895995 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad515b-b744-45b2-a576-3980e7e4df36 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:42.910159 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1283/2534] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:42.914502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad515b-b744-45b2-a576-3980e7e4df36 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:42.924271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad515b-b744-45b2-a576-3980e7e4df36 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:42.929382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ec327-2b22-4cff-9d8d-e9150c523890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:42.930056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ec327-2b22-4cff-9d8d-e9150c523890 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:42.930339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ec327-2b22-4cff-9d8d-e9150c523890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:42.930617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ec327-2b22-4cff-9d8d-e9150c523890 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:42.931299 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ec327-2b22-4cff-9d8d-e9150c523890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:42.933591 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ec327-2b22-4cff-9d8d-e9150c523890 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:42.963144 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1284/2535] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:42 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:42.963328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc83ee-543b-4c7e-a862-8c3b27af0fb1 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:42.964409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ec327-2b22-4cff-9d8d-e9150c523890 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:43.026524 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4842ab-40a5-4a67-baec-85f355ea48fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:43.027094 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4842ab-40a5-4a67-baec-85f355ea48fa service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:43.027374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4842ab-40a5-4a67-baec-85f355ea48fa service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:43.027645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4842ab-40a5-4a67-baec-85f355ea48fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:43.028544 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ec327-2b22-4cff-9d8d-e9150c523890 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:43.029468 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-8a4842ab-40a5-4a67-baec-85f355ea48fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:43.033348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0d6df-a55a-428c-8441-6cfbe7809e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:43.034018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0d6df-a55a-428c-8441-6cfbe7809e6d admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:43.034368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0d6df-a55a-428c-8441-6cfbe7809e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:43.034586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0d6df-a55a-428c-8441-6cfbe7809e6d admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:43.035273 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0d6df-a55a-428c-8441-6cfbe7809e6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:43.037221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0d6df-a55a-428c-8441-6cfbe7809e6d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:43.040675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4842ab-40a5-4a67-baec-85f355ea48fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:43.042294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4842ab-40a5-4a67-baec-85f355ea48fa service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:43.053136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1252/2536] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:11:43.060508 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1285/2537] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:42 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:43.064512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4842ab-40a5-4a67-baec-85f355ea48fa service placement] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:43.075605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4842ab-40a5-4a67-baec-85f355ea48fa service placement] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:43.080425 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da949b-3cb3-4888-91ef-2ba6c074fc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:43.081069 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da949b-3cb3-4888-91ef-2ba6c074fc2a admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:43.082694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da949b-3cb3-4888-91ef-2ba6c074fc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:43.082694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da949b-3cb3-4888-91ef-2ba6c074fc2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:43.082694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da949b-3cb3-4888-91ef-2ba6c074fc2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:43.084291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da949b-3cb3-4888-91ef-2ba6c074fc2a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:43.094549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1253/2538] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:43.098684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0d6df-a55a-428c-8441-6cfbe7809e6d admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:43.109965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0d6df-a55a-428c-8441-6cfbe7809e6d admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:43.115643 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c66152-b066-4f4a-a0aa-72994d93f6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:43.116408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c66152-b066-4f4a-a0aa-72994d93f6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:43.116851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c66152-b066-4f4a-a0aa-72994d93f6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:43.117364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c66152-b066-4f4a-a0aa-72994d93f6e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:43.118195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c66152-b066-4f4a-a0aa-72994d93f6e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:43.120711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c66152-b066-4f4a-a0aa-72994d93f6e8 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:43.132254 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1286/2539] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:43.136521 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da949b-3cb3-4888-91ef-2ba6c074fc2a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:43.146424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da949b-3cb3-4888-91ef-2ba6c074fc2a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:43.150948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd33ca-e94c-4133-84a7-414c1ea0c2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:43.152465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd33ca-e94c-4133-84a7-414c1ea0c2bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:43.152739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd33ca-e94c-4133-84a7-414c1ea0c2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:43.153009 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd33ca-e94c-4133-84a7-414c1ea0c2bf admin admin] PATH_INFO: `/v3/projects/e050184b9afa483cac1b95d4ea19dcaa/users/ff8c7b44db8d495a9ddb0a76d4b8ff83/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:43.169439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd33ca-e94c-4133-84a7-414c1ea0c2bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e050184b9afa483cac1b95d4ea19dcaa, user_id=ff8c7b44db8d495a9ddb0a76d4b8ff83, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:43.172181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd33ca-e94c-4133-84a7-414c1ea0c2bf admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:43.189992 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1254/2540] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:43 2022] PUT /identity/v3/projects/e050184b9afa483cac1b95d4ea19dcaa/users/ff8c7b44db8d495a9ddb0a76d4b8ff83/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:43.202033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9a8b6-de77-45ad-b886-f4aee13ce5aa None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:43.202332 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9a8b6-de77-45ad-b886-f4aee13ce5aa None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:43.202617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9a8b6-de77-45ad-b886-f4aee13ce5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:43.231427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-63b9a8b6-de77-45ad-b886-f4aee13ce5aa None None] MFA Rules not processed for user `ff8c7b44db8d495a9ddb0a76d4b8ff83`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:43.232651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-63b9a8b6-de77-45ad-b886-f4aee13ce5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:43.277531 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:43.300337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1287/2541] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:43.311578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd33ca-e94c-4133-84a7-414c1ea0c2bf admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:43.313304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d5bd33ca-e94c-4133-84a7-414c1ea0c2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:43.371525 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd33ca-e94c-4133-84a7-414c1ea0c2bf admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:43.375258 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8512502-1dfd-46a4-be1d-a3ae4e171cad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:43.375898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8512502-1dfd-46a4-be1d-a3ae4e171cad service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:43.376178 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8512502-1dfd-46a4-be1d-a3ae4e171cad service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:43.376452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8512502-1dfd-46a4-be1d-a3ae4e171cad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:43.384581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8512502-1dfd-46a4-be1d-a3ae4e171cad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:43.386999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8512502-1dfd-46a4-be1d-a3ae4e171cad service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:43.391425 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:43.397221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1255/2542] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:44.750477 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e019cf7-f872-4462-88e1-b6dc4d8e9cc6 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:44.750926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e019cf7-f872-4462-88e1-b6dc4d8e9cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:44.751137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e019cf7-f872-4462-88e1-b6dc4d8e9cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:44.768976 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-0e019cf7-f872-4462-88e1-b6dc4d8e9cc6 None None] MFA Rules not processed for user `ff8c7b44db8d495a9ddb0a76d4b8ff83`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:44.770195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0e019cf7-f872-4462-88e1-b6dc4d8e9cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:44.776937 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1288/2543] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:45.613602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8512502-1dfd-46a4-be1d-a3ae4e171cad service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:45.615199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c8512502-1dfd-46a4-be1d-a3ae4e171cad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:45.684210 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8512502-1dfd-46a4-be1d-a3ae4e171cad service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:45.689176 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:45.689999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:45.690287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:45.690567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] PATH_INFO: `/v3/users/61187cefa09d48bd981bfaebecba7493` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:45.691771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] RBAC: Authorizing `identity:delete_user(user_id=61187cefa09d48bd981bfaebecba7493)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:45.693726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:45.745779 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61187cefa09d48bd981bfaebecba7493'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:45.746443 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61187cefa09d48bd981bfaebecba7493'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:45.753818 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61187cefa09d48bd981bfaebecba7493'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:45.754433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61187cefa09d48bd981bfaebecba7493'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:45.767958 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61187cefa09d48bd981bfaebecba7493'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:45.781013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1256/2544] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:45 2022] DELETE /identity/v3/users/61187cefa09d48bd981bfaebecba7493 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:45.785484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5097d6-a92d-4e49-b72a-eb249f211637 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:45.785966 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5097d6-a92d-4e49-b72a-eb249f211637 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:45.786058 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5097d6-a92d-4e49-b72a-eb249f211637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:45.803245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-da5097d6-a92d-4e49-b72a-eb249f211637 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:45.804462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-da5097d6-a92d-4e49-b72a-eb249f211637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:45.865185 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1289/2545] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:45.874973 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:45.876395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:45.943718 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046dba7-81b2-402c-87c4-35908bb9beac admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:45.947366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c97b23-9319-44b6-9197-bbedba4ddb65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:45.948527 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c97b23-9319-44b6-9197-bbedba4ddb65 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:45.948803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c97b23-9319-44b6-9197-bbedba4ddb65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:45.949103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c97b23-9319-44b6-9197-bbedba4ddb65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:45.958046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c97b23-9319-44b6-9197-bbedba4ddb65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:45.959461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c97b23-9319-44b6-9197-bbedba4ddb65 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:45.969446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1257/2546] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:46.473226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5097d6-a92d-4e49-b72a-eb249f211637 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:46.474842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-da5097d6-a92d-4e49-b72a-eb249f211637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:46.488750 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5097d6-a92d-4e49-b72a-eb249f211637 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:46.493828 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:46.494626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:46.495541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:46.495816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] PATH_INFO: `/v3/projects/3614d0717bc24f6c9002397e09d5a497` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:46.497158 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3614d0717bc24f6c9002397e09d5a497)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:46.499115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:46.558282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3614d0717bc24f6c9002397e09d5a497', 'request_id': 'req-bbc05217-4b83-4439-b213-f42fd069485e'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:46.564685 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3614d0717bc24f6c9002397e09d5a497', 'request_id': 'req-bbc05217-4b83-4439-b213-f42fd069485e'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:46.572470 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3614d0717bc24f6c9002397e09d5a497', 'request_id': 'req-bbc05217-4b83-4439-b213-f42fd069485e'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:46.584078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] The token cache is being invalidate because project 3614d0717bc24f6c9002397e09d5a497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:46.584374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:46.585284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:46.586918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1290/2547] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:46 2022] DELETE /identity/v3/projects/3614d0717bc24f6c9002397e09d5a497 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:46.591447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c97b23-9319-44b6-9197-bbedba4ddb65 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:46.601759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c97b23-9319-44b6-9197-bbedba4ddb65 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:46.611817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:46.663976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2eb02-473b-4482-9929-fa52fcca43d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:46.664839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2eb02-473b-4482-9929-fa52fcca43d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:46.665144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2eb02-473b-4482-9929-fa52fcca43d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:46.665424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2eb02-473b-4482-9929-fa52fcca43d7 admin admin] PATH_INFO: `/v3/users/36b4c1b4856d40c5ba42bd78d90b7c40` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:46.667152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2eb02-473b-4482-9929-fa52fcca43d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36b4c1b4856d40c5ba42bd78d90b7c40)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:46.669073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2eb02-473b-4482-9929-fa52fcca43d7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:46.723247 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a6c2eb02-473b-4482-9929-fa52fcca43d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36b4c1b4856d40c5ba42bd78d90b7c40'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:46.723993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a6c2eb02-473b-4482-9929-fa52fcca43d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36b4c1b4856d40c5ba42bd78d90b7c40'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:46.732910 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a6c2eb02-473b-4482-9929-fa52fcca43d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36b4c1b4856d40c5ba42bd78d90b7c40'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:46.733767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a6c2eb02-473b-4482-9929-fa52fcca43d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36b4c1b4856d40c5ba42bd78d90b7c40'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:46.746262 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a6c2eb02-473b-4482-9929-fa52fcca43d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36b4c1b4856d40c5ba42bd78d90b7c40'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:46.759621 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1258/2548] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:46 2022] DELETE /identity/v3/users/36b4c1b4856d40c5ba42bd78d90b7c40 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:47.032394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:47.034573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:47.054372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc05217-4b83-4439-b213-f42fd069485e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:47.141000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4035d2-9827-4847-8ee7-0b4d5e995339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:47.141923 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4035d2-9827-4847-8ee7-0b4d5e995339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:47.142211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4035d2-9827-4847-8ee7-0b4d5e995339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:47.142490 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4035d2-9827-4847-8ee7-0b4d5e995339 admin admin] PATH_INFO: `/v3/projects/a9b13f26bd10455eb9259a18ae2d9ea5` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:47.143720 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4035d2-9827-4847-8ee7-0b4d5e995339 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9b13f26bd10455eb9259a18ae2d9ea5)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:47.145864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4035d2-9827-4847-8ee7-0b4d5e995339 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:47.155787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:47.201833 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-2e4035d2-9827-4847-8ee7-0b4d5e995339 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9b13f26bd10455eb9259a18ae2d9ea5', 'request_id': 'req-2e4035d2-9827-4847-8ee7-0b4d5e995339'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:47.208524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-2e4035d2-9827-4847-8ee7-0b4d5e995339 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9b13f26bd10455eb9259a18ae2d9ea5', 'request_id': 'req-2e4035d2-9827-4847-8ee7-0b4d5e995339'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:47.217174 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-2e4035d2-9827-4847-8ee7-0b4d5e995339 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9b13f26bd10455eb9259a18ae2d9ea5', 'request_id': 'req-2e4035d2-9827-4847-8ee7-0b4d5e995339'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:47.230190 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-2e4035d2-9827-4847-8ee7-0b4d5e995339 admin admin] The token cache is being invalidate because project a9b13f26bd10455eb9259a18ae2d9ea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:47.230684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-2e4035d2-9827-4847-8ee7-0b4d5e995339 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:47.231467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-2e4035d2-9827-4847-8ee7-0b4d5e995339 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:47.233036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1291/2549] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:47 2022] DELETE /identity/v3/projects/a9b13f26bd10455eb9259a18ae2d9ea5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:47.238931 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48e9ba-5012-424e-90de-f2df0e44179e None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:47.239287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48e9ba-5012-424e-90de-f2df0e44179e None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:47.239660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48e9ba-5012-424e-90de-f2df0e44179e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:47.259861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1f48e9ba-5012-424e-90de-f2df0e44179e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:47.261245 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1f48e9ba-5012-424e-90de-f2df0e44179e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:47.322404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1259/2550] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:47.330382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26d82b-d8f3-4435-9872-67ce6782a8a6 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:47.330681 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26d82b-d8f3-4435-9872-67ce6782a8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:47.330964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26d82b-d8f3-4435-9872-67ce6782a8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:47.350121 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-cf26d82b-d8f3-4435-9872-67ce6782a8a6 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:47.351336 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-cf26d82b-d8f3-4435-9872-67ce6782a8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:47.360115 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1292/2551] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:47.364332 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48e9ba-5012-424e-90de-f2df0e44179e None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:47.374947 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48e9ba-5012-424e-90de-f2df0e44179e None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:47.379685 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50051222-5ce8-477d-a325-8920dc834831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:47.380387 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50051222-5ce8-477d-a325-8920dc834831 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:47.380689 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50051222-5ce8-477d-a325-8920dc834831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:47.380960 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50051222-5ce8-477d-a325-8920dc834831 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:47.381707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50051222-5ce8-477d-a325-8920dc834831 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:47.382013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50051222-5ce8-477d-a325-8920dc834831 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:47.383567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50051222-5ce8-477d-a325-8920dc834831 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:47.395820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1260/2552] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:47.400018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26d82b-d8f3-4435-9872-67ce6782a8a6 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:47.409971 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26d82b-d8f3-4435-9872-67ce6782a8a6 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:47.414818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748e804-74ff-4acb-b4f8-dc93321a747e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:47.415471 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748e804-74ff-4acb-b4f8-dc93321a747e admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:47.415760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748e804-74ff-4acb-b4f8-dc93321a747e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:47.416034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748e804-74ff-4acb-b4f8-dc93321a747e admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:47.417323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748e804-74ff-4acb-b4f8-dc93321a747e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:47.419311 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748e804-74ff-4acb-b4f8-dc93321a747e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:47.445448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1293/2553] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:47 2022] POST /identity/v3/projects => generated 401 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:47.449966 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50051222-5ce8-477d-a325-8920dc834831 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:47.459846 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50051222-5ce8-477d-a325-8920dc834831 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:47.480380 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:47.528300 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccd3af-b189-4df8-a75f-404844fa5a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:47.528977 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccd3af-b189-4df8-a75f-404844fa5a79 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:47.529292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccd3af-b189-4df8-a75f-404844fa5a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:47.529573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccd3af-b189-4df8-a75f-404844fa5a79 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:47.530268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccd3af-b189-4df8-a75f-404844fa5a79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:47.532184 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccd3af-b189-4df8-a75f-404844fa5a79 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:47.556503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1261/2554] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:47 2022] POST /identity/v3/users => generated 301 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:47.558323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748e804-74ff-4acb-b4f8-dc93321a747e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:47.568743 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748e804-74ff-4acb-b4f8-dc93321a747e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:47.573861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094fa21-1be2-4f4c-aff6-304381d6b4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:47.574524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094fa21-1be2-4f4c-aff6-304381d6b4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:47.574805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094fa21-1be2-4f4c-aff6-304381d6b4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:47.575076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094fa21-1be2-4f4c-aff6-304381d6b4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:47.575789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094fa21-1be2-4f4c-aff6-304381d6b4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:47.577443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094fa21-1be2-4f4c-aff6-304381d6b4e1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:47.588084 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1294/2555] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:47.592568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccd3af-b189-4df8-a75f-404844fa5a79 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:47.602993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccd3af-b189-4df8-a75f-404844fa5a79 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:47.607911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11a3d6-3563-4598-8565-d6223fe437bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:47.608564 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11a3d6-3563-4598-8565-d6223fe437bb admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:47.608845 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11a3d6-3563-4598-8565-d6223fe437bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:47.609223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11a3d6-3563-4598-8565-d6223fe437bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:47.609969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11a3d6-3563-4598-8565-d6223fe437bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:47.611560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11a3d6-3563-4598-8565-d6223fe437bb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:47.622084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1262/2556] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:47.626320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094fa21-1be2-4f4c-aff6-304381d6b4e1 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:47.638720 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094fa21-1be2-4f4c-aff6-304381d6b4e1 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:47.643599 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2ecf0-a348-46ba-a7e3-b2a12d7e9ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:47.644590 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2ecf0-a348-46ba-a7e3-b2a12d7e9ad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:47.644939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2ecf0-a348-46ba-a7e3-b2a12d7e9ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:47.645311 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2ecf0-a348-46ba-a7e3-b2a12d7e9ad9 admin admin] PATH_INFO: `/v3/projects/6dfdfe559e8741dbaa6830b2f22367c7/users/d121f64f4f5443779469485da6f6d383/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:47.660027 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2ecf0-a348-46ba-a7e3-b2a12d7e9ad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dfdfe559e8741dbaa6830b2f22367c7, user_id=d121f64f4f5443779469485da6f6d383, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:47.662765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2ecf0-a348-46ba-a7e3-b2a12d7e9ad9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:47.675831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1295/2557] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:47 2022] PUT /identity/v3/projects/6dfdfe559e8741dbaa6830b2f22367c7/users/d121f64f4f5443779469485da6f6d383/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:47.687059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5eacd-be18-4c1f-8d72-e4e063a9bdbb None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:47.687548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5eacd-be18-4c1f-8d72-e4e063a9bdbb None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:47.687638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5eacd-be18-4c1f-8d72-e4e063a9bdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:47.714805 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1da5eacd-be18-4c1f-8d72-e4e063a9bdbb None None] MFA Rules not processed for user `d121f64f4f5443779469485da6f6d383`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:47.717411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1da5eacd-be18-4c1f-8d72-e4e063a9bdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:47.781894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1263/2558] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:47.786356 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2ecf0-a348-46ba-a7e3-b2a12d7e9ad9 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:47.847459 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2ecf0-a348-46ba-a7e3-b2a12d7e9ad9 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:47.851969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f6eb5-076f-4911-9060-1371904aad65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:47.852615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f6eb5-076f-4911-9060-1371904aad65 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:47.852899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f6eb5-076f-4911-9060-1371904aad65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:47.853204 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f6eb5-076f-4911-9060-1371904aad65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:47.853881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f6eb5-076f-4911-9060-1371904aad65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:47.856713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f6eb5-076f-4911-9060-1371904aad65 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:47.881950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1296/2559] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:47 2022] POST /identity/v3/projects => generated 401 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:47.886576 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5eacd-be18-4c1f-8d72-e4e063a9bdbb None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:47.898144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5eacd-be18-4c1f-8d72-e4e063a9bdbb None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:47.953659 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f9314-edd5-4756-a8af-3c030204d63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:47.954322 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f9314-edd5-4756-a8af-3c030204d63a admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:47.954603 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f9314-edd5-4756-a8af-3c030204d63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:47.954874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f9314-edd5-4756-a8af-3c030204d63a admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:47.955545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f9314-edd5-4756-a8af-3c030204d63a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:47.957464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f9314-edd5-4756-a8af-3c030204d63a admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:47.976822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1264/2560] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:47 2022] POST /identity/v3/users => generated 307 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:47.981426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f6eb5-076f-4911-9060-1371904aad65 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:47.991512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f6eb5-076f-4911-9060-1371904aad65 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:47.996816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df02041-0e74-4b2b-b07b-59637a155b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:47.997494 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df02041-0e74-4b2b-b07b-59637a155b51 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:47.997780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df02041-0e74-4b2b-b07b-59637a155b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:47.998057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df02041-0e74-4b2b-b07b-59637a155b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:47.998771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df02041-0e74-4b2b-b07b-59637a155b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:48.000358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df02041-0e74-4b2b-b07b-59637a155b51 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:48.010728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1297/2561] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:11:48.014833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f9314-edd5-4756-a8af-3c030204d63a admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:48.024921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f9314-edd5-4756-a8af-3c030204d63a admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:48.029609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7e2d9-9063-49a4-b01c-b5f30e182bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:48.030487 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7e2d9-9063-49a4-b01c-b5f30e182bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:48.030771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7e2d9-9063-49a4-b01c-b5f30e182bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:48.031052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7e2d9-9063-49a4-b01c-b5f30e182bc5 admin admin] PATH_INFO: `/v3/projects/0c1e441ef6d54973a2e198397d21c970/users/c1ab13afd2cd4496a3bfa15b785a7ac3/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:48.046011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7e2d9-9063-49a4-b01c-b5f30e182bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c1e441ef6d54973a2e198397d21c970, user_id=c1ab13afd2cd4496a3bfa15b785a7ac3, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:48.048676 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7e2d9-9063-49a4-b01c-b5f30e182bc5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:48.050937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:48.063387 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1265/2562] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:48 2022] PUT /identity/v3/projects/0c1e441ef6d54973a2e198397d21c970/users/c1ab13afd2cd4496a3bfa15b785a7ac3/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:48.067472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df02041-0e74-4b2b-b07b-59637a155b51 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:48.074039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:48.078225 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df02041-0e74-4b2b-b07b-59637a155b51 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:48.141775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6127d61-c2e2-4be0-a5af-92642ffe6c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:48.142436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6127d61-c2e2-4be0-a5af-92642ffe6c12 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:48.142717 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6127d61-c2e2-4be0-a5af-92642ffe6c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:48.142992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6127d61-c2e2-4be0-a5af-92642ffe6c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:48.143701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6127d61-c2e2-4be0-a5af-92642ffe6c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:48.145309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6127d61-c2e2-4be0-a5af-92642ffe6c12 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:48.155146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1298/2563] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:48.159248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7e2d9-9063-49a4-b01c-b5f30e182bc5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:48.169933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7e2d9-9063-49a4-b01c-b5f30e182bc5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:48.174458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57233c2-210c-42d1-bc20-461ff5af38a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:48.175330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57233c2-210c-42d1-bc20-461ff5af38a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:48.175617 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57233c2-210c-42d1-bc20-461ff5af38a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:48.175895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57233c2-210c-42d1-bc20-461ff5af38a8 admin admin] PATH_INFO: `/v3/projects/0c1e441ef6d54973a2e198397d21c970/users/c1ab13afd2cd4496a3bfa15b785a7ac3/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:48.177824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57233c2-210c-42d1-bc20-461ff5af38a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c1e441ef6d54973a2e198397d21c970, user_id=c1ab13afd2cd4496a3bfa15b785a7ac3, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:48.180450 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57233c2-210c-42d1-bc20-461ff5af38a8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:48.191212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1266/2564] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:48 2022] PUT /identity/v3/projects/0c1e441ef6d54973a2e198397d21c970/users/c1ab13afd2cd4496a3bfa15b785a7ac3/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:48.202621 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1283e-0bae-43fd-a8d5-21b9c8512a20 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:48.202940 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1283e-0bae-43fd-a8d5-21b9c8512a20 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:48.203224 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1283e-0bae-43fd-a8d5-21b9c8512a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:48.232526 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-13c1283e-0bae-43fd-a8d5-21b9c8512a20 None None] MFA Rules not processed for user `c1ab13afd2cd4496a3bfa15b785a7ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:48.233762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-13c1283e-0bae-43fd-a8d5-21b9c8512a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:48.300242 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1299/2565] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:48 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:48.311092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57233c2-210c-42d1-bc20-461ff5af38a8 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:48.312902 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a57233c2-210c-42d1-bc20-461ff5af38a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:48.374786 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57233c2-210c-42d1-bc20-461ff5af38a8 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:48.378477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23817e-02f2-43b6-8172-ee056512412f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:48.379122 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23817e-02f2-43b6-8172-ee056512412f service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:48.379410 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23817e-02f2-43b6-8172-ee056512412f service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:48.379687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23817e-02f2-43b6-8172-ee056512412f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:48.389881 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23817e-02f2-43b6-8172-ee056512412f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:48.391337 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23817e-02f2-43b6-8172-ee056512412f service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:48.404071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1267/2566] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:48 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:48.482843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1283e-0bae-43fd-a8d5-21b9c8512a20 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:48.492571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1283e-0bae-43fd-a8d5-21b9c8512a20 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:48.496668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d3975-e604-4c01-813a-359f4c65b89b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:48.497427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d3975-e604-4c01-813a-359f4c65b89b service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:48.497715 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d3975-e604-4c01-813a-359f4c65b89b service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:48.497993 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d3975-e604-4c01-813a-359f4c65b89b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:48.558817 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d3975-e604-4c01-813a-359f4c65b89b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:48.560292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d3975-e604-4c01-813a-359f4c65b89b service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:48.569767 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1300/2567] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:48 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:52.689695 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23817e-02f2-43b6-8172-ee056512412f service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:52.751695 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23817e-02f2-43b6-8172-ee056512412f service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:52.756224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:52.757106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:52.757401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:52.757679 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] PATH_INFO: `/v3/users/d121f64f4f5443779469485da6f6d383` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:52.758849 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d121f64f4f5443779469485da6f6d383)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:52.760823 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:52.807988 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd121f64f4f5443779469485da6f6d383'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:52.809656 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd121f64f4f5443779469485da6f6d383'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:52.817123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd121f64f4f5443779469485da6f6d383'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:52.817936 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd121f64f4f5443779469485da6f6d383'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:52.829760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd121f64f4f5443779469485da6f6d383'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:52.842256 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1268/2568] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:52 2022] DELETE /identity/v3/users/d121f64f4f5443779469485da6f6d383 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:52.846965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c45cbf-91bc-4d31-a36c-75fb91ed3094 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:52.847435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c45cbf-91bc-4d31-a36c-75fb91ed3094 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:52.847531 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c45cbf-91bc-4d31-a36c-75fb91ed3094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:52.865013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-41c45cbf-91bc-4d31-a36c-75fb91ed3094 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:52.866251 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-41c45cbf-91bc-4d31-a36c-75fb91ed3094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:52.884699 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:52.925855 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1301/2569] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:52.936181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:52.937601 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:52.942916 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:52.997786 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a9f2e-1176-4950-9ef1-21030f5ffed5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:53.001483 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3fcf7-5bf5-4a48-b15f-54dd1475c122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:53.002127 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3fcf7-5bf5-4a48-b15f-54dd1475c122 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:53.002418 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3fcf7-5bf5-4a48-b15f-54dd1475c122 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:53.002697 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3fcf7-5bf5-4a48-b15f-54dd1475c122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:53.012386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3fcf7-5bf5-4a48-b15f-54dd1475c122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:53.013832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3fcf7-5bf5-4a48-b15f-54dd1475c122 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:53.023327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1269/2570] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:11:53.310167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c45cbf-91bc-4d31-a36c-75fb91ed3094 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:53.311861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-41c45cbf-91bc-4d31-a36c-75fb91ed3094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:53.327786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c45cbf-91bc-4d31-a36c-75fb91ed3094 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:53.332948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:53.334776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:53.335182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:53.335563 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] PATH_INFO: `/v3/projects/6dfdfe559e8741dbaa6830b2f22367c7` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:53.336925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dfdfe559e8741dbaa6830b2f22367c7)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:53.340017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:53.458151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dfdfe559e8741dbaa6830b2f22367c7', 'request_id': 'req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:53.468067 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dfdfe559e8741dbaa6830b2f22367c7', 'request_id': 'req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:53.482000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dfdfe559e8741dbaa6830b2f22367c7', 'request_id': 'req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:53.497056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] The token cache is being invalidate because project 6dfdfe559e8741dbaa6830b2f22367c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:53.497341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:53.498600 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:53.500616 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1302/2571] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:53 2022] DELETE /identity/v3/projects/6dfdfe559e8741dbaa6830b2f22367c7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:53.504702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3fcf7-5bf5-4a48-b15f-54dd1475c122 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:53.516670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3fcf7-5bf5-4a48-b15f-54dd1475c122 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:53.588544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6fbc0-0f27-4fc7-b7d8-d24cde381889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:53.590150 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6fbc0-0f27-4fc7-b7d8-d24cde381889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:53.590441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6fbc0-0f27-4fc7-b7d8-d24cde381889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:53.590818 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6fbc0-0f27-4fc7-b7d8-d24cde381889 admin admin] PATH_INFO: `/v3/users/c1ab13afd2cd4496a3bfa15b785a7ac3` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:53.591941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6fbc0-0f27-4fc7-b7d8-d24cde381889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ab13afd2cd4496a3bfa15b785a7ac3)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:53.593938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6fbc0-0f27-4fc7-b7d8-d24cde381889 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:53.649674 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a9e6fbc0-0f27-4fc7-b7d8-d24cde381889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ab13afd2cd4496a3bfa15b785a7ac3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:53.650347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a9e6fbc0-0f27-4fc7-b7d8-d24cde381889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ab13afd2cd4496a3bfa15b785a7ac3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:53.658699 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a9e6fbc0-0f27-4fc7-b7d8-d24cde381889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ab13afd2cd4496a3bfa15b785a7ac3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:53.659355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a9e6fbc0-0f27-4fc7-b7d8-d24cde381889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ab13afd2cd4496a3bfa15b785a7ac3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:53.672860 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a9e6fbc0-0f27-4fc7-b7d8-d24cde381889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ab13afd2cd4496a3bfa15b785a7ac3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:53.689014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1270/2572] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:11:53 2022] DELETE /identity/v3/users/c1ab13afd2cd4496a3bfa15b785a7ac3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:54.033349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.035336 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:54.049343 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719b5f8-e2a2-40f5-bc91-a6cec1749d02 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.106761 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.107543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.107825 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.108097 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793 admin admin] PATH_INFO: `/v3/projects/0c1e441ef6d54973a2e198397d21c970` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.109307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c1e441ef6d54973a2e198397d21c970)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.111232 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.156260 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c1e441ef6d54973a2e198397d21c970', 'request_id': 'req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:54.157339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:54.164004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c1e441ef6d54973a2e198397d21c970', 'request_id': 'req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:54.172245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c1e441ef6d54973a2e198397d21c970', 'request_id': 'req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:54.182252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793 admin admin] The token cache is being invalidate because project 0c1e441ef6d54973a2e198397d21c970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:11:54.183125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:54.183747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d4f31df6-e18c-4d36-ad8b-de4e5ac93793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:11:54.185303 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1303/2573] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:11:54 2022] DELETE /identity/v3/projects/0c1e441ef6d54973a2e198397d21c970 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:54.190220 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82391c00-3bcb-4eac-b043-1eec87335c0a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.190686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82391c00-3bcb-4eac-b043-1eec87335c0a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.190791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82391c00-3bcb-4eac-b043-1eec87335c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.208233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-82391c00-3bcb-4eac-b043-1eec87335c0a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:54.209609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-82391c00-3bcb-4eac-b043-1eec87335c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:54.229667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:54.271311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1271/2574] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:54.279629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956969f-6647-4ef4-ac94-d060e293476a None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.279916 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956969f-6647-4ef4-ac94-d060e293476a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.280195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956969f-6647-4ef4-ac94-d060e293476a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.298684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e956969f-6647-4ef4-ac94-d060e293476a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:54.299865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e956969f-6647-4ef4-ac94-d060e293476a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:54.307748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1304/2575] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:54.311702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82391c00-3bcb-4eac-b043-1eec87335c0a None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.321277 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82391c00-3bcb-4eac-b043-1eec87335c0a None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.326033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a067b2-4ed9-45a2-aa6e-b60770d08570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.326693 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a067b2-4ed9-45a2-aa6e-b60770d08570 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.326979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a067b2-4ed9-45a2-aa6e-b60770d08570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.327256 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a067b2-4ed9-45a2-aa6e-b60770d08570 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.327973 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a067b2-4ed9-45a2-aa6e-b60770d08570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:11:54.328276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a067b2-4ed9-45a2-aa6e-b60770d08570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.329886 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a067b2-4ed9-45a2-aa6e-b60770d08570 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.341801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1272/2576] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:11:54.346151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956969f-6647-4ef4-ac94-d060e293476a None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.355818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956969f-6647-4ef4-ac94-d060e293476a None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.360650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c6f31-3162-4d08-a6f6-32bdae8a2b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.361330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c6f31-3162-4d08-a6f6-32bdae8a2b7a admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.361613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c6f31-3162-4d08-a6f6-32bdae8a2b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.361879 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c6f31-3162-4d08-a6f6-32bdae8a2b7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.362555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c6f31-3162-4d08-a6f6-32bdae8a2b7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.364447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c6f31-3162-4d08-a6f6-32bdae8a2b7a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.388982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1305/2577] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:54 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:54.393006 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a067b2-4ed9-45a2-aa6e-b60770d08570 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.403468 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a067b2-4ed9-45a2-aa6e-b60770d08570 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.480637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226228b7-1394-4f6d-a708-8fa2670f18fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.481345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226228b7-1394-4f6d-a708-8fa2670f18fb admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.481630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226228b7-1394-4f6d-a708-8fa2670f18fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.481907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226228b7-1394-4f6d-a708-8fa2670f18fb admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.482599 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226228b7-1394-4f6d-a708-8fa2670f18fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.484495 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226228b7-1394-4f6d-a708-8fa2670f18fb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.507902 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1273/2578] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:54 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:54.509721 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c6f31-3162-4d08-a6f6-32bdae8a2b7a admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.520342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c6f31-3162-4d08-a6f6-32bdae8a2b7a admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.526768 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f7d3d-3562-4c9b-91af-5f299e2f73b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.527444 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f7d3d-3562-4c9b-91af-5f299e2f73b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.527726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f7d3d-3562-4c9b-91af-5f299e2f73b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.528004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f7d3d-3562-4c9b-91af-5f299e2f73b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.528732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f7d3d-3562-4c9b-91af-5f299e2f73b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.530371 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f7d3d-3562-4c9b-91af-5f299e2f73b8 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.541505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1306/2579] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:54.547167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226228b7-1394-4f6d-a708-8fa2670f18fb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.560298 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226228b7-1394-4f6d-a708-8fa2670f18fb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.565472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f7bd1-c7ef-446c-b7c3-7c0c7cf6e22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.566218 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f7bd1-c7ef-446c-b7c3-7c0c7cf6e22d admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.566737 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f7bd1-c7ef-446c-b7c3-7c0c7cf6e22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.567013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f7bd1-c7ef-446c-b7c3-7c0c7cf6e22d admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.567889 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f7bd1-c7ef-446c-b7c3-7c0c7cf6e22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.569377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f7bd1-c7ef-446c-b7c3-7c0c7cf6e22d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.580990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1274/2580] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:54.585113 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f7d3d-3562-4c9b-91af-5f299e2f73b8 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.600102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f7d3d-3562-4c9b-91af-5f299e2f73b8 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.606544 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790023b-9e2f-4a23-9042-b63e8c749358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.607974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790023b-9e2f-4a23-9042-b63e8c749358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.608260 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790023b-9e2f-4a23-9042-b63e8c749358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.608541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790023b-9e2f-4a23-9042-b63e8c749358 admin admin] PATH_INFO: `/v3/projects/a314d7fc479b4967b8c666c229445962/users/7442eb5771ed4417b2e985b753a48b97/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.627544 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790023b-9e2f-4a23-9042-b63e8c749358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a314d7fc479b4967b8c666c229445962, user_id=7442eb5771ed4417b2e985b753a48b97, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.630320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790023b-9e2f-4a23-9042-b63e8c749358 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.654044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1307/2581] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:54 2022] PUT /identity/v3/projects/a314d7fc479b4967b8c666c229445962/users/7442eb5771ed4417b2e985b753a48b97/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:54.665497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5eb62-e1a3-4eca-926a-874bbea6d6f6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.670592 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5eb62-e1a3-4eca-926a-874bbea6d6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.670592 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5eb62-e1a3-4eca-926a-874bbea6d6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.695985 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-45e5eb62-e1a3-4eca-926a-874bbea6d6f6 None None] MFA Rules not processed for user `7442eb5771ed4417b2e985b753a48b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:54.697592 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-45e5eb62-e1a3-4eca-926a-874bbea6d6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:54.748409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790023b-9e2f-4a23-9042-b63e8c749358 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.750206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-8790023b-9e2f-4a23-9042-b63e8c749358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:54.767012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1275/2582] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:54.772462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5eb62-e1a3-4eca-926a-874bbea6d6f6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.798736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:54.816602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790023b-9e2f-4a23-9042-b63e8c749358 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.820427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4525c-ca2b-42e8-862d-0c93fcecd292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.821097 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4525c-ca2b-42e8-862d-0c93fcecd292 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.821401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4525c-ca2b-42e8-862d-0c93fcecd292 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.822153 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4525c-ca2b-42e8-862d-0c93fcecd292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.824043 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b2c4525c-ca2b-42e8-862d-0c93fcecd292 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:54.845162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5eb62-e1a3-4eca-926a-874bbea6d6f6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.850068 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a3222-56c9-406c-b772-4677753eba71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.850742 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a3222-56c9-406c-b772-4677753eba71 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.851021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a3222-56c9-406c-b772-4677753eba71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.851393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a3222-56c9-406c-b772-4677753eba71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.852082 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a3222-56c9-406c-b772-4677753eba71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.854015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a3222-56c9-406c-b772-4677753eba71 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.879336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1276/2583] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:11:54 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:11:54.883217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a3222-56c9-406c-b772-4677753eba71 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.914183 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4525c-ca2b-42e8-862d-0c93fcecd292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.915619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4525c-ca2b-42e8-862d-0c93fcecd292 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.932161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:54.937035 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1308/2584] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:11:54.954897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a3222-56c9-406c-b772-4677753eba71 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:54.959557 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755593be-6f81-4387-bd7b-232f8db123fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:54.960224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755593be-6f81-4387-bd7b-232f8db123fe admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:54.960510 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755593be-6f81-4387-bd7b-232f8db123fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:54.960791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755593be-6f81-4387-bd7b-232f8db123fe admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:54.961509 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755593be-6f81-4387-bd7b-232f8db123fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:54.963533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755593be-6f81-4387-bd7b-232f8db123fe admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:54.985124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1277/2585] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:11:54 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:11:54.987905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4525c-ca2b-42e8-862d-0c93fcecd292 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:54.999123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4525c-ca2b-42e8-862d-0c93fcecd292 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:55.003713 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f03d0-94ef-49b4-819c-58a70d9e8769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:55.004348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f03d0-94ef-49b4-819c-58a70d9e8769 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:55.004633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f03d0-94ef-49b4-819c-58a70d9e8769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:55.004939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f03d0-94ef-49b4-819c-58a70d9e8769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:55.005699 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f03d0-94ef-49b4-819c-58a70d9e8769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:55.007290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f03d0-94ef-49b4-819c-58a70d9e8769 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:55.018063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1309/2586] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:55.022155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755593be-6f81-4387-bd7b-232f8db123fe admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:55.032299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755593be-6f81-4387-bd7b-232f8db123fe admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:55.036906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e060b-e5ca-4dac-ba15-7cb582829c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:55.037830 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e060b-e5ca-4dac-ba15-7cb582829c44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:55.038117 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e060b-e5ca-4dac-ba15-7cb582829c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:55.038398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e060b-e5ca-4dac-ba15-7cb582829c44 admin admin] PATH_INFO: `/v3/projects/005f345748ae4bdeb3c10c31816f5ea9/users/51167c876f504c35a67086f747c18112/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:55.053396 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e060b-e5ca-4dac-ba15-7cb582829c44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=005f345748ae4bdeb3c10c31816f5ea9, user_id=51167c876f504c35a67086f747c18112, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:55.056042 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e060b-e5ca-4dac-ba15-7cb582829c44 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:55.069026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1278/2587] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:55 2022] PUT /identity/v3/projects/005f345748ae4bdeb3c10c31816f5ea9/users/51167c876f504c35a67086f747c18112/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:55.073710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f03d0-94ef-49b4-819c-58a70d9e8769 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:55.083529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f03d0-94ef-49b4-819c-58a70d9e8769 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:55.140293 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b56b23-b468-4def-93fa-a7062409eeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:55.140888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b56b23-b468-4def-93fa-a7062409eeba admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:55.141206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b56b23-b468-4def-93fa-a7062409eeba admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:55.141491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b56b23-b468-4def-93fa-a7062409eeba admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:55.142243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b56b23-b468-4def-93fa-a7062409eeba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:55.143839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b56b23-b468-4def-93fa-a7062409eeba admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:55.154424 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1310/2588] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:11:55.158476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e060b-e5ca-4dac-ba15-7cb582829c44 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:55.169233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e060b-e5ca-4dac-ba15-7cb582829c44 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:55.173977 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4d5aa-5cda-4004-8ced-304d4a74b6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:55.174867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4d5aa-5cda-4004-8ced-304d4a74b6ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:55.175155 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4d5aa-5cda-4004-8ced-304d4a74b6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:55.175438 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4d5aa-5cda-4004-8ced-304d4a74b6ff admin admin] PATH_INFO: `/v3/projects/005f345748ae4bdeb3c10c31816f5ea9/users/51167c876f504c35a67086f747c18112/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:55.177461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4d5aa-5cda-4004-8ced-304d4a74b6ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=005f345748ae4bdeb3c10c31816f5ea9, user_id=51167c876f504c35a67086f747c18112, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:55.180112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4d5aa-5cda-4004-8ced-304d4a74b6ff admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:55.193638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1279/2589] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:11:55 2022] PUT /identity/v3/projects/005f345748ae4bdeb3c10c31816f5ea9/users/51167c876f504c35a67086f747c18112/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:11:55.205661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f91a-95d2-4d0c-bce1-5d1a250089f0 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:55.205960 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f91a-95d2-4d0c-bce1-5d1a250089f0 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:55.206245 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f91a-95d2-4d0c-bce1-5d1a250089f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:55.232883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-de76f91a-95d2-4d0c-bce1-5d1a250089f0 None None] MFA Rules not processed for user `51167c876f504c35a67086f747c18112`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:11:55.234141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-de76f91a-95d2-4d0c-bce1-5d1a250089f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:55.302346 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1311/2590] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:11:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:11:55.313833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4d5aa-5cda-4004-8ced-304d4a74b6ff admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:11:55.315553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b5d4d5aa-5cda-4004-8ced-304d4a74b6ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:11:55.372867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4d5aa-5cda-4004-8ced-304d4a74b6ff admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:11:55.376570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960283a0-6f02-4f7a-ba26-39eea23e118c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:11:55.377242 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960283a0-6f02-4f7a-ba26-39eea23e118c service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:11:55.377532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960283a0-6f02-4f7a-ba26-39eea23e118c service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:11:55.377813 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960283a0-6f02-4f7a-ba26-39eea23e118c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:11:55.387091 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:11:55.436344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960283a0-6f02-4f7a-ba26-39eea23e118c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:11:55.437750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960283a0-6f02-4f7a-ba26-39eea23e118c service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:11:55.446705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1280/2591] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:11:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:01.841215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f91a-95d2-4d0c-bce1-5d1a250089f0 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:01.861551 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f91a-95d2-4d0c-bce1-5d1a250089f0 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:01.865243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13893042-3cbf-4661-b6fd-c54e4dd062c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:01.865919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13893042-3cbf-4661-b6fd-c54e4dd062c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:01.866209 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13893042-3cbf-4661-b6fd-c54e4dd062c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:01.866493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13893042-3cbf-4661-b6fd-c54e4dd062c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:01.877328 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13893042-3cbf-4661-b6fd-c54e4dd062c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:01.878758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13893042-3cbf-4661-b6fd-c54e4dd062c7 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:01.892545 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1312/2592] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:12:02.020816 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55f4c8-7c72-459d-bdbc-81967bfdf824 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:02.021392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55f4c8-7c72-459d-bdbc-81967bfdf824 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:02.021561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55f4c8-7c72-459d-bdbc-81967bfdf824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:02.085731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-ef55f4c8-7c72-459d-bdbc-81967bfdf824 None None] MFA Rules not processed for user `26d6eb06b23b4db88261ae58d650736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:02.086963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ef55f4c8-7c72-459d-bdbc-81967bfdf824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:02.171466 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1281/2593] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jun 7 09:12:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:02.227447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13893042-3cbf-4661-b6fd-c54e4dd062c7 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:02.228950 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-13893042-3cbf-4661-b6fd-c54e4dd062c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:02.287362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:02.324265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13893042-3cbf-4661-b6fd-c54e4dd062c7 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:02.329836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9542d-c031-469e-b57d-1fd14178048b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586ddffa77a54521b42b2aa50f16cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:02.330528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9542d-c031-469e-b57d-1fd14178048b service swift] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:02.331055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9542d-c031-469e-b57d-1fd14178048b service swift] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:02.331340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9542d-c031-469e-b57d-1fd14178048b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:02.340756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9542d-c031-469e-b57d-1fd14178048b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:02.342486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9542d-c031-469e-b57d-1fd14178048b service swift] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:02.351078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1313/2594] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jun 7 09:12:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:12:02.872548 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad776ca-7a35-44ba-a5c7-793df508978b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:02.872836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad776ca-7a35-44ba-a5c7-793df508978b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:02.873135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad776ca-7a35-44ba-a5c7-793df508978b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:02.890939 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-dad776ca-7a35-44ba-a5c7-793df508978b None None] MFA Rules not processed for user `26d6eb06b23b4db88261ae58d650736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:02.892144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dad776ca-7a35-44ba-a5c7-793df508978b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:02.899343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1282/2595] 10.222.0.39 () {58 vars in 967 bytes} [Tue Jun 7 09:12:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:02.911430 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9542d-c031-469e-b57d-1fd14178048b service swift] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:02.922569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9542d-c031-469e-b57d-1fd14178048b service swift] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:02.927541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab55472-4ec4-4f6c-b93c-6604caad6101 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586ddffa77a54521b42b2aa50f16cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:02.928227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab55472-4ec4-4f6c-b93c-6604caad6101 service swift] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:02.928514 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab55472-4ec4-4f6c-b93c-6604caad6101 service swift] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:02.928795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab55472-4ec4-4f6c-b93c-6604caad6101 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:02.937070 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab55472-4ec4-4f6c-b93c-6604caad6101 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:02.938687 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab55472-4ec4-4f6c-b93c-6604caad6101 service swift] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:02.946667 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1314/2596] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jun 7 09:12:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:12:03.136389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763eafa-6f08-43b5-932f-607ca0e3b48a None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:03.136874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763eafa-6f08-43b5-932f-607ca0e3b48a None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:03.136979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763eafa-6f08-43b5-932f-607ca0e3b48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:03.154815 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c763eafa-6f08-43b5-932f-607ca0e3b48a None None] MFA Rules not processed for user `26d6eb06b23b4db88261ae58d650736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:03.155995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c763eafa-6f08-43b5-932f-607ca0e3b48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:03.163110 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1283/2597] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jun 7 09:12:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:03.175092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab55472-4ec4-4f6c-b93c-6604caad6101 service swift] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:03.186097 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab55472-4ec4-4f6c-b93c-6604caad6101 service swift] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:03.191127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0020c57-d539-4ae5-b95a-b45753bdd87d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586ddffa77a54521b42b2aa50f16cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:03.191757 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0020c57-d539-4ae5-b95a-b45753bdd87d service swift] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:03.192044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0020c57-d539-4ae5-b95a-b45753bdd87d service swift] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:03.192323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0020c57-d539-4ae5-b95a-b45753bdd87d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:03.200926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0020c57-d539-4ae5-b95a-b45753bdd87d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:03.202596 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0020c57-d539-4ae5-b95a-b45753bdd87d service swift] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:03.211323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1315/2598] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jun 7 09:12:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:12:03.577479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ac9fb-5070-470e-b15d-509982e7585b None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:03.577839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ac9fb-5070-470e-b15d-509982e7585b None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:03.578127 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ac9fb-5070-470e-b15d-509982e7585b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:03.596745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-454ac9fb-5070-470e-b15d-509982e7585b None None] MFA Rules not processed for user `26d6eb06b23b4db88261ae58d650736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:03.598025 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-454ac9fb-5070-470e-b15d-509982e7585b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:03.608746 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1284/2599] 10.222.0.39 () {58 vars in 967 bytes} [Tue Jun 7 09:12:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:03.621688 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0020c57-d539-4ae5-b95a-b45753bdd87d service swift] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:03.633654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0020c57-d539-4ae5-b95a-b45753bdd87d service swift] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:03.641179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e784b-fb54-468a-9b03-5a792d847ac2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586ddffa77a54521b42b2aa50f16cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:03.641944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e784b-fb54-468a-9b03-5a792d847ac2 service swift] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:03.642267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e784b-fb54-468a-9b03-5a792d847ac2 service swift] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:03.642566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e784b-fb54-468a-9b03-5a792d847ac2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:03.654727 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e784b-fb54-468a-9b03-5a792d847ac2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:03.656896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e784b-fb54-468a-9b03-5a792d847ac2 service swift] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:03.666541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1316/2600] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jun 7 09:12:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:12:05.759124 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ac9fb-5070-470e-b15d-509982e7585b None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:05.760852 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-454ac9fb-5070-470e-b15d-509982e7585b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:05.815888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:05.833301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ac9fb-5070-470e-b15d-509982e7585b None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:05.838096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:05.839214 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:05.839500 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:05.839779 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] PATH_INFO: `/v3/users/ff8c7b44db8d495a9ddb0a76d4b8ff83` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:05.841669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff8c7b44db8d495a9ddb0a76d4b8ff83)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:05.843618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:05.896645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff8c7b44db8d495a9ddb0a76d4b8ff83'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:05.897374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff8c7b44db8d495a9ddb0a76d4b8ff83'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:05.907296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff8c7b44db8d495a9ddb0a76d4b8ff83'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:05.907712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff8c7b44db8d495a9ddb0a76d4b8ff83'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:05.922235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff8c7b44db8d495a9ddb0a76d4b8ff83'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:05.935444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1285/2601] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:05 2022] DELETE /identity/v3/users/ff8c7b44db8d495a9ddb0a76d4b8ff83 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:12:05.939566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af973b-8134-4b79-ba7f-4070ba7213a9 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:05.940044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af973b-8134-4b79-ba7f-4070ba7213a9 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:05.940151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af973b-8134-4b79-ba7f-4070ba7213a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:05.958810 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-61af973b-8134-4b79-ba7f-4070ba7213a9 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:05.960016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-61af973b-8134-4b79-ba7f-4070ba7213a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:06.028931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1317/2602] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:06.038985 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:06.040462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:06.103134 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94bfcb-e203-40fb-9b05-9e022445a946 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:06.107452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8883b-a2b3-4d55-a9d6-7bf80f024ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:06.108101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8883b-a2b3-4d55-a9d6-7bf80f024ffc service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:06.108395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8883b-a2b3-4d55-a9d6-7bf80f024ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:06.108671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8883b-a2b3-4d55-a9d6-7bf80f024ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:06.117929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8883b-a2b3-4d55-a9d6-7bf80f024ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:06.119350 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8883b-a2b3-4d55-a9d6-7bf80f024ffc service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:06.130024 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1286/2603] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:06.302668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af973b-8134-4b79-ba7f-4070ba7213a9 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:06.304130 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-61af973b-8134-4b79-ba7f-4070ba7213a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:06.316486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af973b-8134-4b79-ba7f-4070ba7213a9 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:06.321239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8593b5-6a3b-4975-9162-d1aa597cda9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:06.322107 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8593b5-6a3b-4975-9162-d1aa597cda9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:06.322395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8593b5-6a3b-4975-9162-d1aa597cda9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:06.322670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8593b5-6a3b-4975-9162-d1aa597cda9f admin admin] PATH_INFO: `/v3/projects/e050184b9afa483cac1b95d4ea19dcaa` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:06.323871 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8593b5-6a3b-4975-9162-d1aa597cda9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e050184b9afa483cac1b95d4ea19dcaa)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:06.325877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8593b5-6a3b-4975-9162-d1aa597cda9f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:06.375354 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-1b8593b5-6a3b-4975-9162-d1aa597cda9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e050184b9afa483cac1b95d4ea19dcaa', 'request_id': 'req-1b8593b5-6a3b-4975-9162-d1aa597cda9f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:06.381409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-1b8593b5-6a3b-4975-9162-d1aa597cda9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e050184b9afa483cac1b95d4ea19dcaa', 'request_id': 'req-1b8593b5-6a3b-4975-9162-d1aa597cda9f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:06.389322 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-1b8593b5-6a3b-4975-9162-d1aa597cda9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e050184b9afa483cac1b95d4ea19dcaa', 'request_id': 'req-1b8593b5-6a3b-4975-9162-d1aa597cda9f'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:06.399697 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-1b8593b5-6a3b-4975-9162-d1aa597cda9f admin admin] The token cache is being invalidate because project e050184b9afa483cac1b95d4ea19dcaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:06.399989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-1b8593b5-6a3b-4975-9162-d1aa597cda9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:06.400712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-1b8593b5-6a3b-4975-9162-d1aa597cda9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:06.402298 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1318/2604] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:06 2022] DELETE /identity/v3/projects/e050184b9afa483cac1b95d4ea19dcaa => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:06.850467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8883b-a2b3-4d55-a9d6-7bf80f024ffc service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:06.851938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-eba8883b-a2b3-4d55-a9d6-7bf80f024ffc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:06.864963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8883b-a2b3-4d55-a9d6-7bf80f024ffc service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:06.922287 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:06.923162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:06.923444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:06.923721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] PATH_INFO: `/v3/users/7442eb5771ed4417b2e985b753a48b97` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:06.924891 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7442eb5771ed4417b2e985b753a48b97)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:06.926848 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:06.938939 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:06.972861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7442eb5771ed4417b2e985b753a48b97'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:06.973540 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7442eb5771ed4417b2e985b753a48b97'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:06.980448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7442eb5771ed4417b2e985b753a48b97'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:06.981645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7442eb5771ed4417b2e985b753a48b97'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:06.993392 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7442eb5771ed4417b2e985b753a48b97'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:07.005286 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1287/2605] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:06 2022] DELETE /identity/v3/users/7442eb5771ed4417b2e985b753a48b97 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:07.009738 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0ffe3-b960-432f-ac1d-c486258a5afb None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:07.010193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0ffe3-b960-432f-ac1d-c486258a5afb None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:07.010307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0ffe3-b960-432f-ac1d-c486258a5afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:07.016124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:07.029903 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-c5d0ffe3-b960-432f-ac1d-c486258a5afb None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:07.031200 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c5d0ffe3-b960-432f-ac1d-c486258a5afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:07.092285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1319/2606] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:07.102767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:07.104243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:07.163739 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45314655-e1d7-4d0b-bac4-685a1054258c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:07.167381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c99345-e887-4355-b6a8-256753c4ce03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:07.168104 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c99345-e887-4355-b6a8-256753c4ce03 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:07.168393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c99345-e887-4355-b6a8-256753c4ce03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:07.168669 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c99345-e887-4355-b6a8-256753c4ce03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:07.177634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c99345-e887-4355-b6a8-256753c4ce03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:07.179059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c99345-e887-4355-b6a8-256753c4ce03 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:07.189680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1288/2607] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:07.456742 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0ffe3-b960-432f-ac1d-c486258a5afb None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:07.458210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c5d0ffe3-b960-432f-ac1d-c486258a5afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:07.472445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0ffe3-b960-432f-ac1d-c486258a5afb None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:07.477307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:07.478201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:07.478492 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:07.478770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] PATH_INFO: `/v3/projects/a314d7fc479b4967b8c666c229445962` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:07.479968 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a314d7fc479b4967b8c666c229445962)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:07.482734 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:07.529580 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a314d7fc479b4967b8c666c229445962', 'request_id': 'req-fb33a006-7561-421e-92ab-33590e0dbcc5'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:07.535492 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a314d7fc479b4967b8c666c229445962', 'request_id': 'req-fb33a006-7561-421e-92ab-33590e0dbcc5'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:07.543864 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a314d7fc479b4967b8c666c229445962', 'request_id': 'req-fb33a006-7561-421e-92ab-33590e0dbcc5'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:07.556466 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] The token cache is being invalidate because project a314d7fc479b4967b8c666c229445962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:07.556754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:07.557416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:07.559821 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1320/2608] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:07 2022] DELETE /identity/v3/projects/a314d7fc479b4967b8c666c229445962 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:07.563762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c99345-e887-4355-b6a8-256753c4ce03 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:07.573321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c99345-e887-4355-b6a8-256753c4ce03 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:07.629532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a53288-63ee-4d69-9632-1a1439e3e5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:07.631602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a53288-63ee-4d69-9632-1a1439e3e5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:07.632139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a53288-63ee-4d69-9632-1a1439e3e5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:07.632687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a53288-63ee-4d69-9632-1a1439e3e5cb admin admin] PATH_INFO: `/v3/users/51167c876f504c35a67086f747c18112` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:07.634422 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a53288-63ee-4d69-9632-1a1439e3e5cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=51167c876f504c35a67086f747c18112)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:07.638295 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a53288-63ee-4d69-9632-1a1439e3e5cb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:07.679996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:07.689616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d6a53288-63ee-4d69-9632-1a1439e3e5cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51167c876f504c35a67086f747c18112'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:07.690545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d6a53288-63ee-4d69-9632-1a1439e3e5cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51167c876f504c35a67086f747c18112'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:07.697911 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d6a53288-63ee-4d69-9632-1a1439e3e5cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51167c876f504c35a67086f747c18112'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:07.698546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d6a53288-63ee-4d69-9632-1a1439e3e5cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51167c876f504c35a67086f747c18112'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:07.710191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d6a53288-63ee-4d69-9632-1a1439e3e5cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51167c876f504c35a67086f747c18112'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:07.722888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1289/2609] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:07 2022] DELETE /identity/v3/users/51167c876f504c35a67086f747c18112 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:07.993800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:07.995369 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:08.008767 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33a006-7561-421e-92ab-33590e0dbcc5 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:08.071283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:08.072162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:08.072443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:08.072717 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be admin admin] PATH_INFO: `/v3/projects/005f345748ae4bdeb3c10c31816f5ea9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:08.073996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be admin admin] RBAC: Authorizing `identity:delete_project(project_id=005f345748ae4bdeb3c10c31816f5ea9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:08.076602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:08.110735 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:08.127581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '005f345748ae4bdeb3c10c31816f5ea9', 'request_id': 'req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:08.134146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '005f345748ae4bdeb3c10c31816f5ea9', 'request_id': 'req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:08.145624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '005f345748ae4bdeb3c10c31816f5ea9', 'request_id': 'req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:08.161902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be admin admin] The token cache is being invalidate because project 005f345748ae4bdeb3c10c31816f5ea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:08.162194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:08.162837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5785e76d-9e6b-4f69-857a-4da1f3c1a3be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:08.164527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1321/2610] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:07 2022] DELETE /identity/v3/projects/005f345748ae4bdeb3c10c31816f5ea9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:08.170855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2b138-8408-4818-bdb1-960a9545bcea None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:08.171299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2b138-8408-4818-bdb1-960a9545bcea None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:08.171501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2b138-8408-4818-bdb1-960a9545bcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:08.189772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e0d2b138-8408-4818-bdb1-960a9545bcea None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:08.190962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e0d2b138-8408-4818-bdb1-960a9545bcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:08.256792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1290/2611] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:12:08.265543 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163dcfa-bf43-4b26-bc02-424f60fcf655 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:08.265860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163dcfa-bf43-4b26-bc02-424f60fcf655 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:08.266169 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163dcfa-bf43-4b26-bc02-424f60fcf655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:08.284668 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e163dcfa-bf43-4b26-bc02-424f60fcf655 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:08.285887 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e163dcfa-bf43-4b26-bc02-424f60fcf655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:08.293742 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1322/2612] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:08.297845 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2b138-8408-4818-bdb1-960a9545bcea None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:08.307312 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2b138-8408-4818-bdb1-960a9545bcea None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:08.311909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f5db3-21a5-4721-90e2-6ca163cb97d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:08.312549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f5db3-21a5-4721-90e2-6ca163cb97d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:08.312830 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f5db3-21a5-4721-90e2-6ca163cb97d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:08.313126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f5db3-21a5-4721-90e2-6ca163cb97d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:08.313839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f5db3-21a5-4721-90e2-6ca163cb97d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:08.314139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f5db3-21a5-4721-90e2-6ca163cb97d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:08.315715 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f5db3-21a5-4721-90e2-6ca163cb97d5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:08.329838 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1291/2613] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:08.334536 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163dcfa-bf43-4b26-bc02-424f60fcf655 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:08.344625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163dcfa-bf43-4b26-bc02-424f60fcf655 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:08.349552 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afed03-4412-47ca-abf3-ad133c3318fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:08.350191 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afed03-4412-47ca-abf3-ad133c3318fd admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:08.350474 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afed03-4412-47ca-abf3-ad133c3318fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:08.350748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afed03-4412-47ca-abf3-ad133c3318fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:08.351469 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6afed03-4412-47ca-abf3-ad133c3318fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:08.353448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6afed03-4412-47ca-abf3-ad133c3318fd admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:08.379637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1323/2614] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:08 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:08.383732 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f5db3-21a5-4721-90e2-6ca163cb97d5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:08.393918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f5db3-21a5-4721-90e2-6ca163cb97d5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:08.450777 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94204f9d-4bbd-4684-9610-0067312a9a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:08.451446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94204f9d-4bbd-4684-9610-0067312a9a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:08.451725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94204f9d-4bbd-4684-9610-0067312a9a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:08.452025 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94204f9d-4bbd-4684-9610-0067312a9a75 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:08.452705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94204f9d-4bbd-4684-9610-0067312a9a75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:08.454634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94204f9d-4bbd-4684-9610-0067312a9a75 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:08.475748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1292/2615] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:08 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:08.479815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afed03-4412-47ca-abf3-ad133c3318fd admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:08.490437 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afed03-4412-47ca-abf3-ad133c3318fd admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:08.495014 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ea283-fd11-4d84-8d87-d429e693de56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:08.495666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ea283-fd11-4d84-8d87-d429e693de56 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:08.495952 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ea283-fd11-4d84-8d87-d429e693de56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:08.496753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ea283-fd11-4d84-8d87-d429e693de56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:08.497509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ea283-fd11-4d84-8d87-d429e693de56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:08.499090 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ea283-fd11-4d84-8d87-d429e693de56 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:08.509186 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1324/2616] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:08.513421 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94204f9d-4bbd-4684-9610-0067312a9a75 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:08.523233 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94204f9d-4bbd-4684-9610-0067312a9a75 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:08.528070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1437cb-de7b-45bc-81eb-1c1cd420d3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:08.528871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1437cb-de7b-45bc-81eb-1c1cd420d3ec admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:08.529203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1437cb-de7b-45bc-81eb-1c1cd420d3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:08.529485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1437cb-de7b-45bc-81eb-1c1cd420d3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:08.530205 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1437cb-de7b-45bc-81eb-1c1cd420d3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:08.531815 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1437cb-de7b-45bc-81eb-1c1cd420d3ec admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:08.546569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1293/2617] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:08.548537 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ea283-fd11-4d84-8d87-d429e693de56 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:08.561817 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ea283-fd11-4d84-8d87-d429e693de56 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:08.566594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10147a13-09df-4692-85c2-902c21667649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:08.567487 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10147a13-09df-4692-85c2-902c21667649 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:08.567773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10147a13-09df-4692-85c2-902c21667649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:08.568051 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10147a13-09df-4692-85c2-902c21667649 admin admin] PATH_INFO: `/v3/projects/a05df95bc7b444e494aa98b09f6f3e52/users/4cb845105733481e9b3f32d3c8c2c402/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:08.583302 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10147a13-09df-4692-85c2-902c21667649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a05df95bc7b444e494aa98b09f6f3e52, user_id=4cb845105733481e9b3f32d3c8c2c402, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:08.585991 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10147a13-09df-4692-85c2-902c21667649 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:08.599033 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1325/2618] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:08 2022] PUT /identity/v3/projects/a05df95bc7b444e494aa98b09f6f3e52/users/4cb845105733481e9b3f32d3c8c2c402/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:08.604137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386264e3-a93c-4290-bcc2-a4c038466b70 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:08.604479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386264e3-a93c-4290-bcc2-a4c038466b70 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:08.604765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386264e3-a93c-4290-bcc2-a4c038466b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:08.623236 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-386264e3-a93c-4290-bcc2-a4c038466b70 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:08.624453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-386264e3-a93c-4290-bcc2-a4c038466b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:08.636377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:08.686311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1294/2619] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:12:08.695715 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10147a13-09df-4692-85c2-902c21667649 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:08.697440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-10147a13-09df-4692-85c2-902c21667649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:08.755419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10147a13-09df-4692-85c2-902c21667649 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:08.759056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfdef0a-2236-43d3-bf20-bb488df63c36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:08.759688 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdef0a-2236-43d3-bf20-bb488df63c36 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:08.759973 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdef0a-2236-43d3-bf20-bb488df63c36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:08.760247 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdef0a-2236-43d3-bf20-bb488df63c36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:08.769129 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfdef0a-2236-43d3-bf20-bb488df63c36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:08.770553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfdef0a-2236-43d3-bf20-bb488df63c36 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:08.780392 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1326/2620] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:12.764706 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386264e3-a93c-4290-bcc2-a4c038466b70 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:12.774601 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386264e3-a93c-4290-bcc2-a4c038466b70 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:12.778543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77df5b-b2ce-46ff-8b4f-a1e4232a97ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:12.779239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77df5b-b2ce-46ff-8b4f-a1e4232a97ac service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:12.779502 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77df5b-b2ce-46ff-8b4f-a1e4232a97ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:12.779781 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77df5b-b2ce-46ff-8b4f-a1e4232a97ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:12.781698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-2e77df5b-b2ce-46ff-8b4f-a1e4232a97ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:12.795313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77df5b-b2ce-46ff-8b4f-a1e4232a97ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:12.796767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77df5b-b2ce-46ff-8b4f-a1e4232a97ac service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:12.808581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1295/2621] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:13.079063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eef415-e32f-462b-9028-a4ebc4b1f45c None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:13.079555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eef415-e32f-462b-9028-a4ebc4b1f45c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:13.079649 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eef415-e32f-462b-9028-a4ebc4b1f45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:13.111626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-b0eef415-e32f-462b-9028-a4ebc4b1f45c None None] MFA Rules not processed for user `4cb845105733481e9b3f32d3c8c2c402`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:13.113021 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b0eef415-e32f-462b-9028-a4ebc4b1f45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:13.169431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:13.207850 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1327/2622] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:13.212292 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77df5b-b2ce-46ff-8b4f-a1e4232a97ac service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:13.222480 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77df5b-b2ce-46ff-8b4f-a1e4232a97ac service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:13.228330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e747cc-0a06-441f-944c-f79ec44ac9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:13.229016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e747cc-0a06-441f-944c-f79ec44ac9aa admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:13.229388 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e747cc-0a06-441f-944c-f79ec44ac9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:13.229671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e747cc-0a06-441f-944c-f79ec44ac9aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:13.230377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e747cc-0a06-441f-944c-f79ec44ac9aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:13.232328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e747cc-0a06-441f-944c-f79ec44ac9aa admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:13.260582 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1296/2623] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:13 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:13.265023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eef415-e32f-462b-9028-a4ebc4b1f45c None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:13.277048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eef415-e32f-462b-9028-a4ebc4b1f45c None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:13.375858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd0466-5c06-4a93-8db4-6ca3463d746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:13.376562 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd0466-5c06-4a93-8db4-6ca3463d746d admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:13.376843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd0466-5c06-4a93-8db4-6ca3463d746d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:13.377143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd0466-5c06-4a93-8db4-6ca3463d746d admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:13.377833 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd0466-5c06-4a93-8db4-6ca3463d746d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:13.379750 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd0466-5c06-4a93-8db4-6ca3463d746d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:13.409036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1328/2624] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:13 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:13.414220 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e747cc-0a06-441f-944c-f79ec44ac9aa admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:13.426606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e747cc-0a06-441f-944c-f79ec44ac9aa admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:13.431710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d72603-cb90-45f4-82ae-52a38f9464d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:13.432391 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d72603-cb90-45f4-82ae-52a38f9464d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:13.432674 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d72603-cb90-45f4-82ae-52a38f9464d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:13.432953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d72603-cb90-45f4-82ae-52a38f9464d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:13.433913 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d72603-cb90-45f4-82ae-52a38f9464d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:13.435522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d72603-cb90-45f4-82ae-52a38f9464d9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:13.453211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1297/2625] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:13.457202 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd0466-5c06-4a93-8db4-6ca3463d746d admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:13.468545 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd0466-5c06-4a93-8db4-6ca3463d746d admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:13.473813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6034b32-b5a0-4aa7-911b-16a237d5a74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:13.474897 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6034b32-b5a0-4aa7-911b-16a237d5a74b admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:13.475181 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6034b32-b5a0-4aa7-911b-16a237d5a74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:13.475456 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6034b32-b5a0-4aa7-911b-16a237d5a74b admin admin] PATH_INFO: `/v3/projects/55e04881ccc34a06a8d235ad4aa28f33/users/7f78151ceafd4dfab3dca80123671468/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:13.495651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6034b32-b5a0-4aa7-911b-16a237d5a74b admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e04881ccc34a06a8d235ad4aa28f33, user_id=7f78151ceafd4dfab3dca80123671468, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:13.498624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6034b32-b5a0-4aa7-911b-16a237d5a74b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:13.515001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1329/2626] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:13 2022] PUT /identity/v3/projects/55e04881ccc34a06a8d235ad4aa28f33/users/7f78151ceafd4dfab3dca80123671468/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:13.518870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d72603-cb90-45f4-82ae-52a38f9464d9 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:13.529627 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d72603-cb90-45f4-82ae-52a38f9464d9 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:13.606019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2e3af-2e4b-4872-9e1b-ee26718b0b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:13.606792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2e3af-2e4b-4872-9e1b-ee26718b0b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:13.607008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2e3af-2e4b-4872-9e1b-ee26718b0b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:13.607634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2e3af-2e4b-4872-9e1b-ee26718b0b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:13.608010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2e3af-2e4b-4872-9e1b-ee26718b0b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:13.609672 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2e3af-2e4b-4872-9e1b-ee26718b0b03 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:13.617802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6034b32-b5a0-4aa7-911b-16a237d5a74b admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:13.619874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c6034b32-b5a0-4aa7-911b-16a237d5a74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:13.621364 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1298/2627] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:13.624723 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2e3af-2e4b-4872-9e1b-ee26718b0b03 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:13.637010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2e3af-2e4b-4872-9e1b-ee26718b0b03 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:13.644560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54edc2-c92b-4fcd-aa81-7a0ddd5d506e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:13.645691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54edc2-c92b-4fcd-aa81-7a0ddd5d506e admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:13.646055 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54edc2-c92b-4fcd-aa81-7a0ddd5d506e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:13.646409 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54edc2-c92b-4fcd-aa81-7a0ddd5d506e admin admin] PATH_INFO: `/v3/projects/55e04881ccc34a06a8d235ad4aa28f33/users/7f78151ceafd4dfab3dca80123671468/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:13.648905 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a54edc2-c92b-4fcd-aa81-7a0ddd5d506e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e04881ccc34a06a8d235ad4aa28f33, user_id=7f78151ceafd4dfab3dca80123671468, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:13.653738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a54edc2-c92b-4fcd-aa81-7a0ddd5d506e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:13.671530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1299/2628] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:13 2022] PUT /identity/v3/projects/55e04881ccc34a06a8d235ad4aa28f33/users/7f78151ceafd4dfab3dca80123671468/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:12:13.677398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54edc2-c92b-4fcd-aa81-7a0ddd5d506e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:13.680000 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1a54edc2-c92b-4fcd-aa81-7a0ddd5d506e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:13.730614 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6034b32-b5a0-4aa7-911b-16a237d5a74b admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:13.779241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:13.797206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54edc2-c92b-4fcd-aa81-7a0ddd5d506e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:13.808269 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:13.809839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffb874-0cb1-4398-9830-8a9e74f667b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:13.810944 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffb874-0cb1-4398-9830-8a9e74f667b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:13.813457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffb874-0cb1-4398-9830-8a9e74f667b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:13.814089 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffb874-0cb1-4398-9830-8a9e74f667b3 admin admin] PATH_INFO: `/v3/users/5caffe8082c246c7b00f1eaa98154c46` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:13.816051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffb874-0cb1-4398-9830-8a9e74f667b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5caffe8082c246c7b00f1eaa98154c46)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:13.818908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffb874-0cb1-4398-9830-8a9e74f667b3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:13.828832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07aadb8-d8f8-4312-b956-5583e1a5b42b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:13.829775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07aadb8-d8f8-4312-b956-5583e1a5b42b service placement] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:13.830617 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07aadb8-d8f8-4312-b956-5583e1a5b42b service placement] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:13.831063 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07aadb8-d8f8-4312-b956-5583e1a5b42b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:13.834519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d07aadb8-d8f8-4312-b956-5583e1a5b42b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:13.851072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07aadb8-d8f8-4312-b956-5583e1a5b42b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:13.852979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07aadb8-d8f8-4312-b956-5583e1a5b42b service placement] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:13.865234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1330/2629] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:12:13.906217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dffb874-0cb1-4398-9830-8a9e74f667b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5caffe8082c246c7b00f1eaa98154c46'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:13.908377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dffb874-0cb1-4398-9830-8a9e74f667b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5caffe8082c246c7b00f1eaa98154c46'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:13.923467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dffb874-0cb1-4398-9830-8a9e74f667b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5caffe8082c246c7b00f1eaa98154c46'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:13.927615 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dffb874-0cb1-4398-9830-8a9e74f667b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5caffe8082c246c7b00f1eaa98154c46'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:13.961465 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0dffb874-0cb1-4398-9830-8a9e74f667b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5caffe8082c246c7b00f1eaa98154c46'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:13.991912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1300/2630] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:13 2022] DELETE /identity/v3/users/5caffe8082c246c7b00f1eaa98154c46 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:14.366198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07aadb8-d8f8-4312-b956-5583e1a5b42b service placement] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:14.379445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07aadb8-d8f8-4312-b956-5583e1a5b42b service placement] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:14.449777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:14.451648 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:14.452554 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:14.453052 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] PATH_INFO: `/v3/projects/e3b94ee628c242599b57c104cb97bc86` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:14.454691 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3b94ee628c242599b57c104cb97bc86)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:14.457829 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:14.547703 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3b94ee628c242599b57c104cb97bc86', 'request_id': 'req-104688d3-d2a7-4f0a-a4b6-230f15a0db39'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:14.558293 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3b94ee628c242599b57c104cb97bc86', 'request_id': 'req-104688d3-d2a7-4f0a-a4b6-230f15a0db39'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:14.573246 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3b94ee628c242599b57c104cb97bc86', 'request_id': 'req-104688d3-d2a7-4f0a-a4b6-230f15a0db39'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:14.593457 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] The token cache is being invalidate because project e3b94ee628c242599b57c104cb97bc86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:14.594361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:14.596049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:14.599178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1331/2631] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:14 2022] DELETE /identity/v3/projects/e3b94ee628c242599b57c104cb97bc86 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:18.500539 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f13b93-2629-4901-93d6-8def70c79b89 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:18.501018 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f13b93-2629-4901-93d6-8def70c79b89 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:18.501234 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f13b93-2629-4901-93d6-8def70c79b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:18.532491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-10f13b93-2629-4901-93d6-8def70c79b89 None None] MFA Rules not processed for user `7f78151ceafd4dfab3dca80123671468`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:18.533825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-10f13b93-2629-4901-93d6-8def70c79b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:18.611248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1301/2632] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:18.626632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:18.628197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:18.720952 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:18.728051 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104688d3-d2a7-4f0a-a4b6-230f15a0db39 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:18.735778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24c7f8-e2f5-405c-904c-3302f82b7dc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:18.736820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24c7f8-e2f5-405c-904c-3302f82b7dc4 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:18.737198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24c7f8-e2f5-405c-904c-3302f82b7dc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:18.737558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24c7f8-e2f5-405c-904c-3302f82b7dc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:18.739426 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ae24c7f8-e2f5-405c-904c-3302f82b7dc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:18.815012 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24c7f8-e2f5-405c-904c-3302f82b7dc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:18.816620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24c7f8-e2f5-405c-904c-3302f82b7dc4 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:18.827791 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1332/2633] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:19.945511 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04d556-7fac-41cd-824f-2ae39749cad9 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:19.945978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04d556-7fac-41cd-824f-2ae39749cad9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:19.946066 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04d556-7fac-41cd-824f-2ae39749cad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:19.965762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9b04d556-7fac-41cd-824f-2ae39749cad9 None None] MFA Rules not processed for user `4cb845105733481e9b3f32d3c8c2c402`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:19.966967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9b04d556-7fac-41cd-824f-2ae39749cad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:19.974126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1302/2634] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:27.650431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24c7f8-e2f5-405c-904c-3302f82b7dc4 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:27.651964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ae24c7f8-e2f5-405c-904c-3302f82b7dc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:27.709575 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24c7f8-e2f5-405c-904c-3302f82b7dc4 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:27.713350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa285d-6ac3-47cc-9d59-9a70fd5e6804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:27.713999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa285d-6ac3-47cc-9d59-9a70fd5e6804 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:27.714313 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa285d-6ac3-47cc-9d59-9a70fd5e6804 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:27.714625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa285d-6ac3-47cc-9d59-9a70fd5e6804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:27.716372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-91aa285d-6ac3-47cc-9d59-9a70fd5e6804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:27.782142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa285d-6ac3-47cc-9d59-9a70fd5e6804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:27.783587 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa285d-6ac3-47cc-9d59-9a70fd5e6804 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:27.794143 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1333/2635] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:29.959173 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=65,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:31.126138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04d556-7fac-41cd-824f-2ae39749cad9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:31.127593 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9b04d556-7fac-41cd-824f-2ae39749cad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:31.142523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04d556-7fac-41cd-824f-2ae39749cad9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:31.147266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f342ff4-2db6-4d17-ba89-d316d2dddbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:31.148133 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f342ff4-2db6-4d17-ba89-d316d2dddbc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:31.148409 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f342ff4-2db6-4d17-ba89-d316d2dddbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:31.148677 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f342ff4-2db6-4d17-ba89-d316d2dddbc8 admin admin] PATH_INFO: `/v3/users/4f603b289d964ee3ba3931948acf6788` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:31.149873 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f342ff4-2db6-4d17-ba89-d316d2dddbc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f603b289d964ee3ba3931948acf6788)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:31.151901 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f342ff4-2db6-4d17-ba89-d316d2dddbc8 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:31.196954 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3f342ff4-2db6-4d17-ba89-d316d2dddbc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f603b289d964ee3ba3931948acf6788'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:31.197660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3f342ff4-2db6-4d17-ba89-d316d2dddbc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f603b289d964ee3ba3931948acf6788'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:31.205431 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3f342ff4-2db6-4d17-ba89-d316d2dddbc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f603b289d964ee3ba3931948acf6788'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:31.206049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3f342ff4-2db6-4d17-ba89-d316d2dddbc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f603b289d964ee3ba3931948acf6788'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:31.218267 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3f342ff4-2db6-4d17-ba89-d316d2dddbc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f603b289d964ee3ba3931948acf6788'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:31.230768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1303/2636] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:31 2022] DELETE /identity/v3/users/4f603b289d964ee3ba3931948acf6788 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:31.408882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa285d-6ac3-47cc-9d59-9a70fd5e6804 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:31.420165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa285d-6ac3-47cc-9d59-9a70fd5e6804 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:31.472383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:31.483226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621062ce-c443-49e5-9d56-c882d31bf243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:31.484093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621062ce-c443-49e5-9d56-c882d31bf243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:31.484408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621062ce-c443-49e5-9d56-c882d31bf243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:31.484687 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621062ce-c443-49e5-9d56-c882d31bf243 admin admin] PATH_INFO: `/v3/projects/e10990abd8864745b2766df2f30a4bc8` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:31.485931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621062ce-c443-49e5-9d56-c882d31bf243 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e10990abd8864745b2766df2f30a4bc8)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:31.487884 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621062ce-c443-49e5-9d56-c882d31bf243 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:31.536227 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-621062ce-c443-49e5-9d56-c882d31bf243 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e10990abd8864745b2766df2f30a4bc8', 'request_id': 'req-621062ce-c443-49e5-9d56-c882d31bf243'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:31.542557 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-621062ce-c443-49e5-9d56-c882d31bf243 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e10990abd8864745b2766df2f30a4bc8', 'request_id': 'req-621062ce-c443-49e5-9d56-c882d31bf243'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:31.551922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-621062ce-c443-49e5-9d56-c882d31bf243 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e10990abd8864745b2766df2f30a4bc8', 'request_id': 'req-621062ce-c443-49e5-9d56-c882d31bf243'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:31.563676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-621062ce-c443-49e5-9d56-c882d31bf243 admin admin] The token cache is being invalidate because project e10990abd8864745b2766df2f30a4bc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:31.563962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-621062ce-c443-49e5-9d56-c882d31bf243 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:31.564585 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-621062ce-c443-49e5-9d56-c882d31bf243 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:31.566185 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1334/2637] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:31 2022] DELETE /identity/v3/projects/e10990abd8864745b2766df2f30a4bc8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:31.571145 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a28c-4470-4327-becc-1fd6bbab34d8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:31.571459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a28c-4470-4327-becc-1fd6bbab34d8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:31.571738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a28c-4470-4327-becc-1fd6bbab34d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:31.590824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-74b9a28c-4470-4327-becc-1fd6bbab34d8 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:31.592025 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-74b9a28c-4470-4327-becc-1fd6bbab34d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:31.649181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1304/2638] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:31.657419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdf953-77fe-4819-b700-a85c417b5a4c None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:31.657714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdf953-77fe-4819-b700-a85c417b5a4c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:31.657989 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdf953-77fe-4819-b700-a85c417b5a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:31.676003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-84cdf953-77fe-4819-b700-a85c417b5a4c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:31.677221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-84cdf953-77fe-4819-b700-a85c417b5a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:31.685454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1335/2639] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:31.689501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9a28c-4470-4327-becc-1fd6bbab34d8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:31.699985 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9a28c-4470-4327-becc-1fd6bbab34d8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:31.704714 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5def6f-90e1-4d48-b48b-7b19edf4d572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:31.705414 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5def6f-90e1-4d48-b48b-7b19edf4d572 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:31.705700 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5def6f-90e1-4d48-b48b-7b19edf4d572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:31.705980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5def6f-90e1-4d48-b48b-7b19edf4d572 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:31.706702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5def6f-90e1-4d48-b48b-7b19edf4d572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:31.707008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5def6f-90e1-4d48-b48b-7b19edf4d572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:31.708628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5def6f-90e1-4d48-b48b-7b19edf4d572 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:31.720345 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1305/2640] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:31.724776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdf953-77fe-4819-b700-a85c417b5a4c None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:31.734183 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdf953-77fe-4819-b700-a85c417b5a4c None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:31.738890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1781c-e3ef-4cd8-968a-15cbf338654f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:31.739518 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1781c-e3ef-4cd8-968a-15cbf338654f admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:31.739806 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1781c-e3ef-4cd8-968a-15cbf338654f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:31.740083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1781c-e3ef-4cd8-968a-15cbf338654f admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:31.740777 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1781c-e3ef-4cd8-968a-15cbf338654f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:31.742721 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1781c-e3ef-4cd8-968a-15cbf338654f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:31.769039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1336/2641] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:31 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:31.773072 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5def6f-90e1-4d48-b48b-7b19edf4d572 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:31.783028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5def6f-90e1-4d48-b48b-7b19edf4d572 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:31.841536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594467b-4ba3-4206-a6c0-d02b39207147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:31.842208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594467b-4ba3-4206-a6c0-d02b39207147 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:31.842495 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594467b-4ba3-4206-a6c0-d02b39207147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:31.842771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594467b-4ba3-4206-a6c0-d02b39207147 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:31.843464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594467b-4ba3-4206-a6c0-d02b39207147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:31.845653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594467b-4ba3-4206-a6c0-d02b39207147 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:31.852205 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:31.867019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1306/2642] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:31 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:12:31.871594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1781c-e3ef-4cd8-968a-15cbf338654f admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:31.882018 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1781c-e3ef-4cd8-968a-15cbf338654f admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:31.887693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9cc0f-0d65-4479-8ab9-33a16c5194bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:31.888781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9cc0f-0d65-4479-8ab9-33a16c5194bb admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:31.889309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9cc0f-0d65-4479-8ab9-33a16c5194bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:31.889807 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9cc0f-0d65-4479-8ab9-33a16c5194bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:31.890553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9cc0f-0d65-4479-8ab9-33a16c5194bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:31.892454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9cc0f-0d65-4479-8ab9-33a16c5194bb admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:31.904548 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1337/2643] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:31.908713 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594467b-4ba3-4206-a6c0-d02b39207147 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:31.919000 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594467b-4ba3-4206-a6c0-d02b39207147 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:31.923714 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d0633-f0ac-4832-ade7-6ad894e3f30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:31.924370 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d0633-f0ac-4832-ade7-6ad894e3f30e admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:31.924658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d0633-f0ac-4832-ade7-6ad894e3f30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:31.924935 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d0633-f0ac-4832-ade7-6ad894e3f30e admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:31.925730 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d0633-f0ac-4832-ade7-6ad894e3f30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:31.927332 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d0633-f0ac-4832-ade7-6ad894e3f30e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:31.939018 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1307/2644] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:12:31.943137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9cc0f-0d65-4479-8ab9-33a16c5194bb admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:31.955695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9cc0f-0d65-4479-8ab9-33a16c5194bb admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:31.960517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed98b8a-2613-4a3e-929d-430539b9c254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:31.961425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed98b8a-2613-4a3e-929d-430539b9c254 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:31.961716 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed98b8a-2613-4a3e-929d-430539b9c254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:31.961996 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed98b8a-2613-4a3e-929d-430539b9c254 admin admin] PATH_INFO: `/v3/projects/03f3e6ee68114ed7bd3e7254bfb9b8d9/users/71a3b38c2a6f4247942b5bd6d94becd9/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:31.989964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed98b8a-2613-4a3e-929d-430539b9c254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03f3e6ee68114ed7bd3e7254bfb9b8d9, user_id=71a3b38c2a6f4247942b5bd6d94becd9, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:31.993272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed98b8a-2613-4a3e-929d-430539b9c254 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:32.014053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1338/2645] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:31 2022] PUT /identity/v3/projects/03f3e6ee68114ed7bd3e7254bfb9b8d9/users/71a3b38c2a6f4247942b5bd6d94becd9/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:32.023766 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d7cb7-d264-49a2-9abf-0888efd35fa5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:32.024066 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d7cb7-d264-49a2-9abf-0888efd35fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:32.024348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d7cb7-d264-49a2-9abf-0888efd35fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:32.052743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-540d7cb7-d264-49a2-9abf-0888efd35fa5 None None] MFA Rules not processed for user `71a3b38c2a6f4247942b5bd6d94becd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:32.054279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-540d7cb7-d264-49a2-9abf-0888efd35fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:32.119004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1308/2646] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:32 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:32.123509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed98b8a-2613-4a3e-929d-430539b9c254 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:32.185038 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed98b8a-2613-4a3e-929d-430539b9c254 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:32.189641 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9106a56-4c96-46bf-82a9-489eee560fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:32.190291 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9106a56-4c96-46bf-82a9-489eee560fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:32.190574 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9106a56-4c96-46bf-82a9-489eee560fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:32.190848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9106a56-4c96-46bf-82a9-489eee560fa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:32.191542 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9106a56-4c96-46bf-82a9-489eee560fa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:32.193484 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9106a56-4c96-46bf-82a9-489eee560fa0 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:32.217967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1339/2647] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:32 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:32.222198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d7cb7-d264-49a2-9abf-0888efd35fa5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:32.233685 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d7cb7-d264-49a2-9abf-0888efd35fa5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:32.290876 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4676808-41e1-4611-9c57-d41e90f9507f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:32.291561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4676808-41e1-4611-9c57-d41e90f9507f admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:32.291849 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4676808-41e1-4611-9c57-d41e90f9507f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:32.292654 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4676808-41e1-4611-9c57-d41e90f9507f admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:32.293430 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4676808-41e1-4611-9c57-d41e90f9507f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:32.295349 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4676808-41e1-4611-9c57-d41e90f9507f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:32.318112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1309/2648] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:32 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:32.321910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9106a56-4c96-46bf-82a9-489eee560fa0 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:32.327535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:32.334161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9106a56-4c96-46bf-82a9-489eee560fa0 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:32.339959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f51b7-bba9-42cc-96fc-b6ec2606100c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:32.340609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f51b7-bba9-42cc-96fc-b6ec2606100c admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:32.340895 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f51b7-bba9-42cc-96fc-b6ec2606100c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:32.341203 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f51b7-bba9-42cc-96fc-b6ec2606100c admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:32.341919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f51b7-bba9-42cc-96fc-b6ec2606100c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:32.343524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f51b7-bba9-42cc-96fc-b6ec2606100c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:32.354132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1340/2649] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:32.358825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4676808-41e1-4611-9c57-d41e90f9507f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:32.368718 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4676808-41e1-4611-9c57-d41e90f9507f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:32.373559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e83d5-c0be-4c62-bec6-696c0df75f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:32.374424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e83d5-c0be-4c62-bec6-696c0df75f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:32.374708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e83d5-c0be-4c62-bec6-696c0df75f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:32.375013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e83d5-c0be-4c62-bec6-696c0df75f6f admin admin] PATH_INFO: `/v3/projects/e09cbb89ea3b44a9877c7f36444c5c8f/users/2433e6dfefaf4bffaf279c5caa5cafbe/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:32.390545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e83d5-c0be-4c62-bec6-696c0df75f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e09cbb89ea3b44a9877c7f36444c5c8f, user_id=2433e6dfefaf4bffaf279c5caa5cafbe, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:32.393241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e83d5-c0be-4c62-bec6-696c0df75f6f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:32.406744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1310/2650] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:32 2022] PUT /identity/v3/projects/e09cbb89ea3b44a9877c7f36444c5c8f/users/2433e6dfefaf4bffaf279c5caa5cafbe/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:32.410603 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f51b7-bba9-42cc-96fc-b6ec2606100c admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:32.421219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f51b7-bba9-42cc-96fc-b6ec2606100c admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:32.478786 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf30ee4-74e6-4adf-8a98-df911eea0dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:32.479376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf30ee4-74e6-4adf-8a98-df911eea0dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:32.479729 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf30ee4-74e6-4adf-8a98-df911eea0dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:32.480215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf30ee4-74e6-4adf-8a98-df911eea0dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:32.481023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf30ee4-74e6-4adf-8a98-df911eea0dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:32.482662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf30ee4-74e6-4adf-8a98-df911eea0dc1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:32.492348 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1341/2651] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:12:32.497004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e83d5-c0be-4c62-bec6-696c0df75f6f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:32.507139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e83d5-c0be-4c62-bec6-696c0df75f6f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:32.512463 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ac064-5cd0-45ec-897e-a45e12b1b755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:32.513355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ac064-5cd0-45ec-897e-a45e12b1b755 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:32.513641 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ac064-5cd0-45ec-897e-a45e12b1b755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:32.513918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ac064-5cd0-45ec-897e-a45e12b1b755 admin admin] PATH_INFO: `/v3/projects/e09cbb89ea3b44a9877c7f36444c5c8f/users/2433e6dfefaf4bffaf279c5caa5cafbe/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:32.515880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ac064-5cd0-45ec-897e-a45e12b1b755 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e09cbb89ea3b44a9877c7f36444c5c8f, user_id=2433e6dfefaf4bffaf279c5caa5cafbe, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:32.518530 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ac064-5cd0-45ec-897e-a45e12b1b755 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:32.528528 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1311/2652] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:32 2022] PUT /identity/v3/projects/e09cbb89ea3b44a9877c7f36444c5c8f/users/2433e6dfefaf4bffaf279c5caa5cafbe/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:32.585421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263df017-177e-45c8-8737-d7d6b3e82d03 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:32.585741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263df017-177e-45c8-8737-d7d6b3e82d03 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:32.586131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263df017-177e-45c8-8737-d7d6b3e82d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:32.612397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-263df017-177e-45c8-8737-d7d6b3e82d03 None None] MFA Rules not processed for user `2433e6dfefaf4bffaf279c5caa5cafbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:32.613650 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-263df017-177e-45c8-8737-d7d6b3e82d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:32.684618 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1342/2653] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:32.695872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ac064-5cd0-45ec-897e-a45e12b1b755 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:32.697690 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3f4ac064-5cd0-45ec-897e-a45e12b1b755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:32.721019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:32.756558 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ac064-5cd0-45ec-897e-a45e12b1b755 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:32.760118 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ae00f-3186-483f-8d55-93b80ce5c9ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:32.760756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ae00f-3186-483f-8d55-93b80ce5c9ab service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:32.761681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ae00f-3186-483f-8d55-93b80ce5c9ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:32.761962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ae00f-3186-483f-8d55-93b80ce5c9ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:32.771041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ae00f-3186-483f-8d55-93b80ce5c9ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:32.772455 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ae00f-3186-483f-8d55-93b80ce5c9ab service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:32.782058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1312/2654] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:32.851840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263df017-177e-45c8-8737-d7d6b3e82d03 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:32.862141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263df017-177e-45c8-8737-d7d6b3e82d03 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:32.865755 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64021f06-8ef2-4ab3-851e-69158a85f633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:32.866387 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64021f06-8ef2-4ab3-851e-69158a85f633 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:32.866671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64021f06-8ef2-4ab3-851e-69158a85f633 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:32.866948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64021f06-8ef2-4ab3-851e-69158a85f633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:32.924235 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64021f06-8ef2-4ab3-851e-69158a85f633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:32.925711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64021f06-8ef2-4ab3-851e-69158a85f633 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:32.935046 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1343/2655] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:32 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:35.668912 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ae00f-3186-483f-8d55-93b80ce5c9ab service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:35.733598 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ae00f-3186-483f-8d55-93b80ce5c9ab service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:35.738224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:35.739146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:35.739431 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:35.739712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] PATH_INFO: `/v3/users/71a3b38c2a6f4247942b5bd6d94becd9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:35.740896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=71a3b38c2a6f4247942b5bd6d94becd9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:35.742825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:35.792008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71a3b38c2a6f4247942b5bd6d94becd9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:35.793262 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71a3b38c2a6f4247942b5bd6d94becd9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:35.801373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71a3b38c2a6f4247942b5bd6d94becd9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:35.802097 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71a3b38c2a6f4247942b5bd6d94becd9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:35.814865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71a3b38c2a6f4247942b5bd6d94becd9'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:35.827863 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1313/2656] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:35 2022] DELETE /identity/v3/users/71a3b38c2a6f4247942b5bd6d94becd9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:35.832859 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf833e7-0692-4c89-9dd8-37bcdfc6a1c8 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:35.833369 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf833e7-0692-4c89-9dd8-37bcdfc6a1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:35.833450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf833e7-0692-4c89-9dd8-37bcdfc6a1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:35.851047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-1cf833e7-0692-4c89-9dd8-37bcdfc6a1c8 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:35.852358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1cf833e7-0692-4c89-9dd8-37bcdfc6a1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:35.858714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=100 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:35.937111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1344/2657] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:35.948048 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:35.949893 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:36.002246 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:36.029736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a714a7-05c9-4d75-b522-fbc435c8a4ae admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:36.033459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343910b-52a2-4de2-956b-07be08274d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:36.034112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343910b-52a2-4de2-956b-07be08274d22 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:36.034395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343910b-52a2-4de2-956b-07be08274d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:36.034668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343910b-52a2-4de2-956b-07be08274d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:36.045045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343910b-52a2-4de2-956b-07be08274d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:36.046546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343910b-52a2-4de2-956b-07be08274d22 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:36.058003 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1314/2658] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:36.593439 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf833e7-0692-4c89-9dd8-37bcdfc6a1c8 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:36.594939 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-1cf833e7-0692-4c89-9dd8-37bcdfc6a1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:36.608189 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf833e7-0692-4c89-9dd8-37bcdfc6a1c8 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:36.613102 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:36.614032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:36.614386 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:36.614794 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] PATH_INFO: `/v3/projects/03f3e6ee68114ed7bd3e7254bfb9b8d9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:36.615865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03f3e6ee68114ed7bd3e7254bfb9b8d9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:36.617815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:36.664197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03f3e6ee68114ed7bd3e7254bfb9b8d9', 'request_id': 'req-a95f3307-4b28-4e6d-b2da-01d81937ea48'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:36.670978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03f3e6ee68114ed7bd3e7254bfb9b8d9', 'request_id': 'req-a95f3307-4b28-4e6d-b2da-01d81937ea48'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:36.678138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03f3e6ee68114ed7bd3e7254bfb9b8d9', 'request_id': 'req-a95f3307-4b28-4e6d-b2da-01d81937ea48'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:36.688618 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] The token cache is being invalidate because project 03f3e6ee68114ed7bd3e7254bfb9b8d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:36.688906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:36.689560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:36.691093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1345/2659] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:36 2022] DELETE /identity/v3/projects/03f3e6ee68114ed7bd3e7254bfb9b8d9 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:12:36.695010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343910b-52a2-4de2-956b-07be08274d22 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:36.705104 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343910b-52a2-4de2-956b-07be08274d22 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:36.762161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68e4fb-daf5-4b91-b826-2c0af0725207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:36.763045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68e4fb-daf5-4b91-b826-2c0af0725207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:36.763333 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68e4fb-daf5-4b91-b826-2c0af0725207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:36.763610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68e4fb-daf5-4b91-b826-2c0af0725207 admin admin] PATH_INFO: `/v3/users/2433e6dfefaf4bffaf279c5caa5cafbe` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:36.764900 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68e4fb-daf5-4b91-b826-2c0af0725207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2433e6dfefaf4bffaf279c5caa5cafbe)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:36.767169 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68e4fb-daf5-4b91-b826-2c0af0725207 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:36.822330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0f68e4fb-daf5-4b91-b826-2c0af0725207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2433e6dfefaf4bffaf279c5caa5cafbe'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:36.823204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0f68e4fb-daf5-4b91-b826-2c0af0725207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2433e6dfefaf4bffaf279c5caa5cafbe'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:36.833021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0f68e4fb-daf5-4b91-b826-2c0af0725207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2433e6dfefaf4bffaf279c5caa5cafbe'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:36.833838 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0f68e4fb-daf5-4b91-b826-2c0af0725207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2433e6dfefaf4bffaf279c5caa5cafbe'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:36.847448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0f68e4fb-daf5-4b91-b826-2c0af0725207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2433e6dfefaf4bffaf279c5caa5cafbe'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:36.861639 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1315/2660] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:36 2022] DELETE /identity/v3/users/2433e6dfefaf4bffaf279c5caa5cafbe => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:37.197948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:37.200629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:37.216279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f3307-4b28-4e6d-b2da-01d81937ea48 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:37.286500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88994a2d-a032-4d5b-9bf6-72134db98fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:37.287432 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88994a2d-a032-4d5b-9bf6-72134db98fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:37.287723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88994a2d-a032-4d5b-9bf6-72134db98fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:37.288000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88994a2d-a032-4d5b-9bf6-72134db98fcc admin admin] PATH_INFO: `/v3/projects/e09cbb89ea3b44a9877c7f36444c5c8f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:37.289281 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88994a2d-a032-4d5b-9bf6-72134db98fcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e09cbb89ea3b44a9877c7f36444c5c8f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:37.291414 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88994a2d-a032-4d5b-9bf6-72134db98fcc admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:37.346637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:37.368077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-88994a2d-a032-4d5b-9bf6-72134db98fcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e09cbb89ea3b44a9877c7f36444c5c8f', 'request_id': 'req-88994a2d-a032-4d5b-9bf6-72134db98fcc'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:37.378906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-88994a2d-a032-4d5b-9bf6-72134db98fcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e09cbb89ea3b44a9877c7f36444c5c8f', 'request_id': 'req-88994a2d-a032-4d5b-9bf6-72134db98fcc'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:37.394860 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-88994a2d-a032-4d5b-9bf6-72134db98fcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e09cbb89ea3b44a9877c7f36444c5c8f', 'request_id': 'req-88994a2d-a032-4d5b-9bf6-72134db98fcc'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:37.409904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-88994a2d-a032-4d5b-9bf6-72134db98fcc admin admin] The token cache is being invalidate because project e09cbb89ea3b44a9877c7f36444c5c8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:37.411258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-88994a2d-a032-4d5b-9bf6-72134db98fcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:37.411938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-88994a2d-a032-4d5b-9bf6-72134db98fcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:37.413760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1346/2661] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:37 2022] DELETE /identity/v3/projects/e09cbb89ea3b44a9877c7f36444c5c8f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:37.419108 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec180e-aba8-426a-a7a7-81d1c8752bc6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:37.419395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec180e-aba8-426a-a7a7-81d1c8752bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:37.419675 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec180e-aba8-426a-a7a7-81d1c8752bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:37.438395 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-bbec180e-aba8-426a-a7a7-81d1c8752bc6 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:37.439635 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-bbec180e-aba8-426a-a7a7-81d1c8752bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:37.491280 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:37.505352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1316/2662] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:37.514093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e6bac-a36b-4059-aa92-9bfb3476a731 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:37.514658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e6bac-a36b-4059-aa92-9bfb3476a731 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:37.514984 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e6bac-a36b-4059-aa92-9bfb3476a731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:37.536132 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-c08e6bac-a36b-4059-aa92-9bfb3476a731 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:37.537286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c08e6bac-a36b-4059-aa92-9bfb3476a731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:37.549221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1347/2663] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:37.551213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec180e-aba8-426a-a7a7-81d1c8752bc6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:37.565145 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec180e-aba8-426a-a7a7-81d1c8752bc6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:37.570955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a016753-6ece-4e52-820b-a1829761b483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:37.571713 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a016753-6ece-4e52-820b-a1829761b483 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:37.572001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a016753-6ece-4e52-820b-a1829761b483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:37.572277 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a016753-6ece-4e52-820b-a1829761b483 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:37.573168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a016753-6ece-4e52-820b-a1829761b483 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:37.574050 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a016753-6ece-4e52-820b-a1829761b483 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:37.575700 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a016753-6ece-4e52-820b-a1829761b483 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:37.591331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1317/2664] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:37.595887 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e6bac-a36b-4059-aa92-9bfb3476a731 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:37.607463 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e6bac-a36b-4059-aa92-9bfb3476a731 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:37.613488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa13f19-ad3d-4d46-a3f6-9e2aefdb00f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:37.614775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa13f19-ad3d-4d46-a3f6-9e2aefdb00f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:37.614775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa13f19-ad3d-4d46-a3f6-9e2aefdb00f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:37.614775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa13f19-ad3d-4d46-a3f6-9e2aefdb00f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:37.615448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa13f19-ad3d-4d46-a3f6-9e2aefdb00f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:37.617470 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa13f19-ad3d-4d46-a3f6-9e2aefdb00f2 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:37.662946 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1348/2665] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:37 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:12:37.666953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a016753-6ece-4e52-820b-a1829761b483 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:37.678884 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a016753-6ece-4e52-820b-a1829761b483 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:37.748721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51aa70-a9e5-4706-9e80-1a43b9a0cc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:37.749542 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51aa70-a9e5-4706-9e80-1a43b9a0cc6e admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:37.749902 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51aa70-a9e5-4706-9e80-1a43b9a0cc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:37.750248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51aa70-a9e5-4706-9e80-1a43b9a0cc6e admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:37.751182 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51aa70-a9e5-4706-9e80-1a43b9a0cc6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:37.753325 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51aa70-a9e5-4706-9e80-1a43b9a0cc6e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:37.782285 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1318/2666] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:37 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:37.783809 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa13f19-ad3d-4d46-a3f6-9e2aefdb00f2 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:37.795205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa13f19-ad3d-4d46-a3f6-9e2aefdb00f2 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:37.800431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bd469-3ff0-4523-8275-beff347146e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:37.801163 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bd469-3ff0-4523-8275-beff347146e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:37.801409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bd469-3ff0-4523-8275-beff347146e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:37.801719 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bd469-3ff0-4523-8275-beff347146e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:37.802453 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bd469-3ff0-4523-8275-beff347146e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:37.804060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bd469-3ff0-4523-8275-beff347146e3 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:37.818758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1349/2667] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:37.820218 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51aa70-a9e5-4706-9e80-1a43b9a0cc6e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:37.830768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51aa70-a9e5-4706-9e80-1a43b9a0cc6e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:37.835920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf90508-bc70-474e-bcde-e0c264111763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:37.836730 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf90508-bc70-474e-bcde-e0c264111763 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:37.837020 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf90508-bc70-474e-bcde-e0c264111763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:37.837324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf90508-bc70-474e-bcde-e0c264111763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:37.838049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf90508-bc70-474e-bcde-e0c264111763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:37.841149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf90508-bc70-474e-bcde-e0c264111763 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:37.854055 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1319/2668] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:37.859068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bd469-3ff0-4523-8275-beff347146e3 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:37.870559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bd469-3ff0-4523-8275-beff347146e3 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:37.877795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54651857-428c-44e2-ab7d-252a71f48b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:37.878840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54651857-428c-44e2-ab7d-252a71f48b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:37.879228 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54651857-428c-44e2-ab7d-252a71f48b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:37.880687 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54651857-428c-44e2-ab7d-252a71f48b0f admin admin] PATH_INFO: `/v3/projects/452a1fe8f17e4b53b0de42c0c997e718/users/27cc96e8b31640cf8f35b0a4f47f4a60/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:37.906443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54651857-428c-44e2-ab7d-252a71f48b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=452a1fe8f17e4b53b0de42c0c997e718, user_id=27cc96e8b31640cf8f35b0a4f47f4a60, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:37.909254 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54651857-428c-44e2-ab7d-252a71f48b0f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:37.929128 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1350/2669] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:37 2022] PUT /identity/v3/projects/452a1fe8f17e4b53b0de42c0c997e718/users/27cc96e8b31640cf8f35b0a4f47f4a60/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:12:37.936639 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4ee5d-1f18-4f4b-9c91-e98e35fb51b6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:37.936934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4ee5d-1f18-4f4b-9c91-e98e35fb51b6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:37.937257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4ee5d-1f18-4f4b-9c91-e98e35fb51b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:37.965326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-23b4ee5d-1f18-4f4b-9c91-e98e35fb51b6 None None] MFA Rules not processed for user `27cc96e8b31640cf8f35b0a4f47f4a60`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:37.967257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-23b4ee5d-1f18-4f4b-9c91-e98e35fb51b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:38.034062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1320/2670] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:38.038432 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54651857-428c-44e2-ab7d-252a71f48b0f admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:38.111830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54651857-428c-44e2-ab7d-252a71f48b0f admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:38.116819 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f6544-7f66-4eb8-b594-6673d5d85c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:38.117522 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f6544-7f66-4eb8-b594-6673d5d85c0e admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:38.118188 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f6544-7f66-4eb8-b594-6673d5d85c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:38.118476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f6544-7f66-4eb8-b594-6673d5d85c0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:38.119171 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f6544-7f66-4eb8-b594-6673d5d85c0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:38.122020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f6544-7f66-4eb8-b594-6673d5d85c0e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:38.159461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1351/2671] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:38 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:38.160796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4ee5d-1f18-4f4b-9c91-e98e35fb51b6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:38.174677 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4ee5d-1f18-4f4b-9c91-e98e35fb51b6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:38.238953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:38.258075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e779b6-1f86-456a-90e1-0015c2f1c3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:38.259626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e779b6-1f86-456a-90e1-0015c2f1c3d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:38.260039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e779b6-1f86-456a-90e1-0015c2f1c3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:38.260411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e779b6-1f86-456a-90e1-0015c2f1c3d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:38.261870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e779b6-1f86-456a-90e1-0015c2f1c3d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:38.264244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e779b6-1f86-456a-90e1-0015c2f1c3d9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:38.291245 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1321/2672] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:38 2022] POST /identity/v3/users => generated 299 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:38.295305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f6544-7f66-4eb8-b594-6673d5d85c0e admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:38.309207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f6544-7f66-4eb8-b594-6673d5d85c0e admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:38.315129 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531f827-403f-4bf5-b76d-3bd5a4459d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:38.316026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531f827-403f-4bf5-b76d-3bd5a4459d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:38.316633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531f827-403f-4bf5-b76d-3bd5a4459d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:38.317103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531f827-403f-4bf5-b76d-3bd5a4459d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:38.318611 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531f827-403f-4bf5-b76d-3bd5a4459d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:38.320602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531f827-403f-4bf5-b76d-3bd5a4459d9c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:38.338473 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1352/2673] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:38.340432 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e779b6-1f86-456a-90e1-0015c2f1c3d9 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:38.351580 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e779b6-1f86-456a-90e1-0015c2f1c3d9 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:38.357439 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dde227-ea65-41d5-9c96-edf12e3729d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:38.358849 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dde227-ea65-41d5-9c96-edf12e3729d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:38.359307 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dde227-ea65-41d5-9c96-edf12e3729d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:38.359758 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dde227-ea65-41d5-9c96-edf12e3729d9 admin admin] PATH_INFO: `/v3/projects/8ed1167b883647f9a0a6e2e16c0a54cb/users/0687d8e5b1e84a908b8ec85f9133e336/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:38.377939 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dde227-ea65-41d5-9c96-edf12e3729d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed1167b883647f9a0a6e2e16c0a54cb, user_id=0687d8e5b1e84a908b8ec85f9133e336, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:38.380543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dde227-ea65-41d5-9c96-edf12e3729d9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:38.394588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1322/2674] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:38 2022] PUT /identity/v3/projects/8ed1167b883647f9a0a6e2e16c0a54cb/users/0687d8e5b1e84a908b8ec85f9133e336/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:38.405710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531f827-403f-4bf5-b76d-3bd5a4459d9c admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:38.417514 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531f827-403f-4bf5-b76d-3bd5a4459d9c admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:38.463294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:38.508359 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c73ef-ad2b-4851-a80d-74011b986d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:38.510373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c73ef-ad2b-4851-a80d-74011b986d4b admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:38.511054 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c73ef-ad2b-4851-a80d-74011b986d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:38.511257 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c73ef-ad2b-4851-a80d-74011b986d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:38.512014 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c73ef-ad2b-4851-a80d-74011b986d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:38.514877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c73ef-ad2b-4851-a80d-74011b986d4b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:38.525556 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1353/2675] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:38.530299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dde227-ea65-41d5-9c96-edf12e3729d9 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:38.546707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dde227-ea65-41d5-9c96-edf12e3729d9 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:38.554053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35a0e4-71bd-4608-a065-40caf3be0851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:38.554983 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35a0e4-71bd-4608-a065-40caf3be0851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:38.555243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35a0e4-71bd-4608-a065-40caf3be0851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:38.555554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35a0e4-71bd-4608-a065-40caf3be0851 admin admin] PATH_INFO: `/v3/projects/8ed1167b883647f9a0a6e2e16c0a54cb/users/0687d8e5b1e84a908b8ec85f9133e336/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:38.558764 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35a0e4-71bd-4608-a065-40caf3be0851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed1167b883647f9a0a6e2e16c0a54cb, user_id=0687d8e5b1e84a908b8ec85f9133e336, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:38.562402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35a0e4-71bd-4608-a065-40caf3be0851 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:38.578026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1323/2676] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:38 2022] PUT /identity/v3/projects/8ed1167b883647f9a0a6e2e16c0a54cb/users/0687d8e5b1e84a908b8ec85f9133e336/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:38.588682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dc07c-3794-44a0-ad1f-31a4d2863c00 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:38.589999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dc07c-3794-44a0-ad1f-31a4d2863c00 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:38.590376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dc07c-3794-44a0-ad1f-31a4d2863c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:38.606111 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35a0e4-71bd-4608-a065-40caf3be0851 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:38.608416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1a35a0e4-71bd-4608-a065-40caf3be0851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:38.645554 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e83dc07c-3794-44a0-ad1f-31a4d2863c00 None None] MFA Rules not processed for user `0687d8e5b1e84a908b8ec85f9133e336`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:38.646804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e83dc07c-3794-44a0-ad1f-31a4d2863c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:38.695551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35a0e4-71bd-4608-a065-40caf3be0851 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:38.699434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcabeec-0967-49a5-a199-19fc55375002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:38.700571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcabeec-0967-49a5-a199-19fc55375002 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:38.701041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcabeec-0967-49a5-a199-19fc55375002 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:38.701361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcabeec-0967-49a5-a199-19fc55375002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:38.703197 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ebcabeec-0967-49a5-a199-19fc55375002 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:38.715097 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcabeec-0967-49a5-a199-19fc55375002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:38.716473 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcabeec-0967-49a5-a199-19fc55375002 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:38.727574 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1324/2677] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:12:38.734176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1354/2678] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:38.741803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcabeec-0967-49a5-a199-19fc55375002 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:38.743206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ebcabeec-0967-49a5-a199-19fc55375002 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:38.806782 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcabeec-0967-49a5-a199-19fc55375002 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:38.810667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3b082-16e2-4a27-97ba-a57a9509a13a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:38.811402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3b082-16e2-4a27-97ba-a57a9509a13a service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:38.811865 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3b082-16e2-4a27-97ba-a57a9509a13a service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:38.812150 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3b082-16e2-4a27-97ba-a57a9509a13a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:38.821358 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3b082-16e2-4a27-97ba-a57a9509a13a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:38.822891 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3b082-16e2-4a27-97ba-a57a9509a13a service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:38.833027 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1325/2679] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:38.900001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dc07c-3794-44a0-ad1f-31a4d2863c00 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:38.964075 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dc07c-3794-44a0-ad1f-31a4d2863c00 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:38.968758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:38.969686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:38.969971 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:38.970244 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] PATH_INFO: `/v3/users/27cc96e8b31640cf8f35b0a4f47f4a60` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:38.971705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27cc96e8b31640cf8f35b0a4f47f4a60)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:38.973665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:39.024350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27cc96e8b31640cf8f35b0a4f47f4a60'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.024838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27cc96e8b31640cf8f35b0a4f47f4a60'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.032351 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27cc96e8b31640cf8f35b0a4f47f4a60'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.034223 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:39.046410 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27cc96e8b31640cf8f35b0a4f47f4a60'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.047198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27cc96e8b31640cf8f35b0a4f47f4a60'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.061520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1355/2680] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:38 2022] DELETE /identity/v3/users/27cc96e8b31640cf8f35b0a4f47f4a60 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:39.065867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fea3e3-898a-433c-90d9-ea3f61fe7be0 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:39.066156 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fea3e3-898a-433c-90d9-ea3f61fe7be0 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:39.066435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fea3e3-898a-433c-90d9-ea3f61fe7be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:39.086516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-88fea3e3-898a-433c-90d9-ea3f61fe7be0 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:39.087753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-88fea3e3-898a-433c-90d9-ea3f61fe7be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:39.131712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:39.149606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1326/2681] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:39.159641 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:39.161054 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:39.217540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9110f-0fca-4404-a590-9820f85bc2c7 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:39.221428 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a617f-b7ef-4749-865c-4503c9a8bfa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:39.222060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a617f-b7ef-4749-865c-4503c9a8bfa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:39.222341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a617f-b7ef-4749-865c-4503c9a8bfa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:39.222612 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a617f-b7ef-4749-865c-4503c9a8bfa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:39.232272 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a617f-b7ef-4749-865c-4503c9a8bfa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:39.233725 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a617f-b7ef-4749-865c-4503c9a8bfa7 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:39.243047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1356/2682] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:39.690044 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fea3e3-898a-433c-90d9-ea3f61fe7be0 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:39.692293 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-88fea3e3-898a-433c-90d9-ea3f61fe7be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:39.708348 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fea3e3-898a-433c-90d9-ea3f61fe7be0 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:39.714576 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:39.715756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:39.716248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:39.716667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] PATH_INFO: `/v3/projects/452a1fe8f17e4b53b0de42c0c997e718` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:39.718198 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=452a1fe8f17e4b53b0de42c0c997e718)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:39.732327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:39.807517 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '452a1fe8f17e4b53b0de42c0c997e718', 'request_id': 'req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.813713 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '452a1fe8f17e4b53b0de42c0c997e718', 'request_id': 'req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.822855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '452a1fe8f17e4b53b0de42c0c997e718', 'request_id': 'req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.843056 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] The token cache is being invalidate because project 452a1fe8f17e4b53b0de42c0c997e718 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:39.843353 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.844021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.845855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1327/2683] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:39 2022] DELETE /identity/v3/projects/452a1fe8f17e4b53b0de42c0c997e718 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:39.849753 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a617f-b7ef-4749-865c-4503c9a8bfa7 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:39.863036 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a617f-b7ef-4749-865c-4503c9a8bfa7 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:39.925695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7681f50-f1a3-4c83-b246-1a4f1e9d3293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:39.926800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7681f50-f1a3-4c83-b246-1a4f1e9d3293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:39.927088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7681f50-f1a3-4c83-b246-1a4f1e9d3293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:39.927369 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7681f50-f1a3-4c83-b246-1a4f1e9d3293 admin admin] PATH_INFO: `/v3/users/0687d8e5b1e84a908b8ec85f9133e336` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:39.928592 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7681f50-f1a3-4c83-b246-1a4f1e9d3293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0687d8e5b1e84a908b8ec85f9133e336)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:39.930654 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7681f50-f1a3-4c83-b246-1a4f1e9d3293 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:39.976569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d7681f50-f1a3-4c83-b246-1a4f1e9d3293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0687d8e5b1e84a908b8ec85f9133e336'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.977261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d7681f50-f1a3-4c83-b246-1a4f1e9d3293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0687d8e5b1e84a908b8ec85f9133e336'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.984943 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d7681f50-f1a3-4c83-b246-1a4f1e9d3293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0687d8e5b1e84a908b8ec85f9133e336'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.998629 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d7681f50-f1a3-4c83-b246-1a4f1e9d3293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0687d8e5b1e84a908b8ec85f9133e336'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:39.999440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-d7681f50-f1a3-4c83-b246-1a4f1e9d3293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0687d8e5b1e84a908b8ec85f9133e336'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:40.008969 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:40.014789 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1357/2684] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:39 2022] DELETE /identity/v3/users/0687d8e5b1e84a908b8ec85f9133e336 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:40.287544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:40.289010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:40.302444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c532b2-6310-4f55-bc2f-a4c72f23b5bd admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:40.364291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:40.365191 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:40.365482 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:40.365760 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6 admin admin] PATH_INFO: `/v3/projects/8ed1167b883647f9a0a6e2e16c0a54cb` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:40.366933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ed1167b883647f9a0a6e2e16c0a54cb)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:40.368814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:40.414207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed1167b883647f9a0a6e2e16c0a54cb', 'request_id': 'req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:40.419522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed1167b883647f9a0a6e2e16c0a54cb', 'request_id': 'req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:40.426779 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed1167b883647f9a0a6e2e16c0a54cb', 'request_id': 'req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:40.434431 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:40.438602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6 admin admin] The token cache is being invalidate because project 8ed1167b883647f9a0a6e2e16c0a54cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:40.438899 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:40.440009 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-85a4a55f-8fc8-4e5e-bbf9-a1e8537454c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:40.441851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1328/2685] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:40 2022] DELETE /identity/v3/projects/8ed1167b883647f9a0a6e2e16c0a54cb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:40.446842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590008f-3c14-4c97-a8f9-bf639ccb9200 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:40.447154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590008f-3c14-4c97-a8f9-bf639ccb9200 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:40.447403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590008f-3c14-4c97-a8f9-bf639ccb9200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:40.466150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-a590008f-3c14-4c97-a8f9-bf639ccb9200 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:40.467347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a590008f-3c14-4c97-a8f9-bf639ccb9200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:40.528706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1358/2686] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:40.536290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5437555-9e95-41da-80a7-102552a5b919 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:40.536580 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5437555-9e95-41da-80a7-102552a5b919 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:40.536856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5437555-9e95-41da-80a7-102552a5b919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:40.556918 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a5437555-9e95-41da-80a7-102552a5b919 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:40.558457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a5437555-9e95-41da-80a7-102552a5b919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:40.566698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1329/2687] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:40.570893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590008f-3c14-4c97-a8f9-bf639ccb9200 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:40.581381 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590008f-3c14-4c97-a8f9-bf639ccb9200 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:40.585974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ee8c6-f894-49f1-b29b-c36e27df0d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:40.586622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ee8c6-f894-49f1-b29b-c36e27df0d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:40.586908 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ee8c6-f894-49f1-b29b-c36e27df0d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:40.587184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ee8c6-f894-49f1-b29b-c36e27df0d67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:40.587900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ee8c6-f894-49f1-b29b-c36e27df0d67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:40.588253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ee8c6-f894-49f1-b29b-c36e27df0d67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:40.590559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ee8c6-f894-49f1-b29b-c36e27df0d67 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:40.602214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1359/2688] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:40.606641 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5437555-9e95-41da-80a7-102552a5b919 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:40.615805 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5437555-9e95-41da-80a7-102552a5b919 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:40.620684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc63eed-8f8d-4220-8937-76f0e6f75415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:40.621467 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc63eed-8f8d-4220-8937-76f0e6f75415 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:40.621752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc63eed-8f8d-4220-8937-76f0e6f75415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:40.622027 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc63eed-8f8d-4220-8937-76f0e6f75415 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:40.622727 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc63eed-8f8d-4220-8937-76f0e6f75415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:40.624602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc63eed-8f8d-4220-8937-76f0e6f75415 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:40.650157 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1330/2689] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:40 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:40.654581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ee8c6-f894-49f1-b29b-c36e27df0d67 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:40.664493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ee8c6-f894-49f1-b29b-c36e27df0d67 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:40.719419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cde5d-f729-4ded-84a4-aa23b8e265c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:40.720097 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cde5d-f729-4ded-84a4-aa23b8e265c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:40.720381 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cde5d-f729-4ded-84a4-aa23b8e265c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:40.720904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cde5d-f729-4ded-84a4-aa23b8e265c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:40.721607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cde5d-f729-4ded-84a4-aa23b8e265c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:40.723710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cde5d-f729-4ded-84a4-aa23b8e265c4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:40.765709 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1360/2690] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:40 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:40.769359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc63eed-8f8d-4220-8937-76f0e6f75415 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:40.779869 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc63eed-8f8d-4220-8937-76f0e6f75415 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:40.784491 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec6f0e-6003-4543-aa14-0ea71d5e0464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:40.785181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec6f0e-6003-4543-aa14-0ea71d5e0464 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:40.785470 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec6f0e-6003-4543-aa14-0ea71d5e0464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:40.785750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec6f0e-6003-4543-aa14-0ea71d5e0464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:40.786498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec6f0e-6003-4543-aa14-0ea71d5e0464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:40.788086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec6f0e-6003-4543-aa14-0ea71d5e0464 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:40.799086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1331/2691] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:40.803141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cde5d-f729-4ded-84a4-aa23b8e265c4 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:40.814078 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cde5d-f729-4ded-84a4-aa23b8e265c4 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:40.818700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5532d4-3e86-4c6f-b7b9-22f954382e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:40.819341 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5532d4-3e86-4c6f-b7b9-22f954382e86 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:40.819627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5532d4-3e86-4c6f-b7b9-22f954382e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:40.819900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5532d4-3e86-4c6f-b7b9-22f954382e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:40.820652 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5532d4-3e86-4c6f-b7b9-22f954382e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:40.822305 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5532d4-3e86-4c6f-b7b9-22f954382e86 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:40.833166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1361/2692] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:40.837256 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec6f0e-6003-4543-aa14-0ea71d5e0464 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:40.847581 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec6f0e-6003-4543-aa14-0ea71d5e0464 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:40.852248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752676f3-dbaf-48dd-98da-9036bc9f7472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:40.853151 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752676f3-dbaf-48dd-98da-9036bc9f7472 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:40.853432 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752676f3-dbaf-48dd-98da-9036bc9f7472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:40.853707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752676f3-dbaf-48dd-98da-9036bc9f7472 admin admin] PATH_INFO: `/v3/projects/60cf417b548a43d6817f075661cf131d/users/6d10372faa5e4f77b23ff9acce4578e7/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:40.869468 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752676f3-dbaf-48dd-98da-9036bc9f7472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60cf417b548a43d6817f075661cf131d, user_id=6d10372faa5e4f77b23ff9acce4578e7, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:40.873904 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752676f3-dbaf-48dd-98da-9036bc9f7472 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:40.887138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1332/2693] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:40 2022] PUT /identity/v3/projects/60cf417b548a43d6817f075661cf131d/users/6d10372faa5e4f77b23ff9acce4578e7/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:40.900524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a941-2a0c-4e20-9db0-771784cd8148 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:40.900911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a941-2a0c-4e20-9db0-771784cd8148 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:40.901502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a941-2a0c-4e20-9db0-771784cd8148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:40.930017 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-3214a941-2a0c-4e20-9db0-771784cd8148 None None] MFA Rules not processed for user `6d10372faa5e4f77b23ff9acce4578e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:40.931270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-3214a941-2a0c-4e20-9db0-771784cd8148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:40.987915 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:41.000723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1362/2694] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:41.005142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752676f3-dbaf-48dd-98da-9036bc9f7472 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:41.086175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752676f3-dbaf-48dd-98da-9036bc9f7472 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:41.094792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f5a87-6762-4196-8fbc-db4de872785c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:41.095710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f5a87-6762-4196-8fbc-db4de872785c admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:41.095995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f5a87-6762-4196-8fbc-db4de872785c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:41.097769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f5a87-6762-4196-8fbc-db4de872785c admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:41.097769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f5a87-6762-4196-8fbc-db4de872785c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:41.104440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f5a87-6762-4196-8fbc-db4de872785c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:41.134035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1333/2695] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:41 2022] POST /identity/v3/projects => generated 391 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:41.138562 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214a941-2a0c-4e20-9db0-771784cd8148 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:41.149655 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214a941-2a0c-4e20-9db0-771784cd8148 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:41.206436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f0f5f-6fea-4dbb-bca7-96ee98c8dbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:41.207041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f0f5f-6fea-4dbb-bca7-96ee98c8dbbd admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:41.207319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f0f5f-6fea-4dbb-bca7-96ee98c8dbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:41.207598 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f0f5f-6fea-4dbb-bca7-96ee98c8dbbd admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:41.208292 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f0f5f-6fea-4dbb-bca7-96ee98c8dbbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:41.210273 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f0f5f-6fea-4dbb-bca7-96ee98c8dbbd admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:41.231221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1363/2696] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:41 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:41.235097 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f5a87-6762-4196-8fbc-db4de872785c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:41.245351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f5a87-6762-4196-8fbc-db4de872785c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:41.249893 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81cde1-62d1-4ab5-a49c-153e9a44e235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:41.250540 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81cde1-62d1-4ab5-a49c-153e9a44e235 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:41.250819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81cde1-62d1-4ab5-a49c-153e9a44e235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:41.251093 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81cde1-62d1-4ab5-a49c-153e9a44e235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:41.251803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81cde1-62d1-4ab5-a49c-153e9a44e235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:41.253402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81cde1-62d1-4ab5-a49c-153e9a44e235 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:41.264255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1334/2697] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:12:41.268427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f0f5f-6fea-4dbb-bca7-96ee98c8dbbd admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:41.279265 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f0f5f-6fea-4dbb-bca7-96ee98c8dbbd admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:41.283902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76508f62-d3b2-4de1-9682-40e94a0a4053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:41.284775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76508f62-d3b2-4de1-9682-40e94a0a4053 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:41.285057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76508f62-d3b2-4de1-9682-40e94a0a4053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:41.285373 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76508f62-d3b2-4de1-9682-40e94a0a4053 admin admin] PATH_INFO: `/v3/projects/f25f197fdb4e4e7eb69ba7bc06a59c4b/users/dcd0c068368443d0a1d35ac5e1ea642a/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:41.299906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76508f62-d3b2-4de1-9682-40e94a0a4053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f25f197fdb4e4e7eb69ba7bc06a59c4b, user_id=dcd0c068368443d0a1d35ac5e1ea642a, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:41.303005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76508f62-d3b2-4de1-9682-40e94a0a4053 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:41.315693 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1364/2698] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:41 2022] PUT /identity/v3/projects/f25f197fdb4e4e7eb69ba7bc06a59c4b/users/dcd0c068368443d0a1d35ac5e1ea642a/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:41.319694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81cde1-62d1-4ab5-a49c-153e9a44e235 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:41.330308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81cde1-62d1-4ab5-a49c-153e9a44e235 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:41.388604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc59a7-6733-4101-9ce3-4999c06b6b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:41.389342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc59a7-6733-4101-9ce3-4999c06b6b48 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:41.389633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc59a7-6733-4101-9ce3-4999c06b6b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:41.390008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc59a7-6733-4101-9ce3-4999c06b6b48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:41.390765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc59a7-6733-4101-9ce3-4999c06b6b48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:41.392571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc59a7-6733-4101-9ce3-4999c06b6b48 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:41.403365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1335/2699] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:41.407638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76508f62-d3b2-4de1-9682-40e94a0a4053 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:41.418364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76508f62-d3b2-4de1-9682-40e94a0a4053 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:41.434368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3222ed1-fd5b-4615-a982-45d4ded5e989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:41.435284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3222ed1-fd5b-4615-a982-45d4ded5e989 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:41.435573 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3222ed1-fd5b-4615-a982-45d4ded5e989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:41.435851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3222ed1-fd5b-4615-a982-45d4ded5e989 admin admin] PATH_INFO: `/v3/projects/f25f197fdb4e4e7eb69ba7bc06a59c4b/users/dcd0c068368443d0a1d35ac5e1ea642a/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:41.437883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3222ed1-fd5b-4615-a982-45d4ded5e989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f25f197fdb4e4e7eb69ba7bc06a59c4b, user_id=dcd0c068368443d0a1d35ac5e1ea642a, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:41.440823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3222ed1-fd5b-4615-a982-45d4ded5e989 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:41.452940 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1365/2700] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:41 2022] PUT /identity/v3/projects/f25f197fdb4e4e7eb69ba7bc06a59c4b/users/dcd0c068368443d0a1d35ac5e1ea642a/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:41.464204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac14467-7e54-4b49-bd27-8b58f89b9648 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:41.464667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac14467-7e54-4b49-bd27-8b58f89b9648 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:41.464981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac14467-7e54-4b49-bd27-8b58f89b9648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:41.477909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:41.492969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4ac14467-7e54-4b49-bd27-8b58f89b9648 None None] MFA Rules not processed for user `dcd0c068368443d0a1d35ac5e1ea642a`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:41.494227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4ac14467-7e54-4b49-bd27-8b58f89b9648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:41.567710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1336/2701] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:41.579111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3222ed1-fd5b-4615-a982-45d4ded5e989 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:41.580813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d3222ed1-fd5b-4615-a982-45d4ded5e989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:41.637555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3222ed1-fd5b-4615-a982-45d4ded5e989 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:41.641128 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1c2a3-0359-48eb-969c-111033af6a71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:41.641766 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1c2a3-0359-48eb-969c-111033af6a71 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:41.642048 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1c2a3-0359-48eb-969c-111033af6a71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:41.642321 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1c2a3-0359-48eb-969c-111033af6a71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:41.651108 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1c2a3-0359-48eb-969c-111033af6a71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:41.653046 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1c2a3-0359-48eb-969c-111033af6a71 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:41.662284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1366/2702] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:41.706013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac14467-7e54-4b49-bd27-8b58f89b9648 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:41.707423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4ac14467-7e54-4b49-bd27-8b58f89b9648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:41.720060 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac14467-7e54-4b49-bd27-8b58f89b9648 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:41.723840 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab46973-56c1-4e64-9f50-5f43dfd40a39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:41.724479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab46973-56c1-4e64-9f50-5f43dfd40a39 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:41.724766 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab46973-56c1-4e64-9f50-5f43dfd40a39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:41.725046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab46973-56c1-4e64-9f50-5f43dfd40a39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:41.788875 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab46973-56c1-4e64-9f50-5f43dfd40a39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:41.790342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab46973-56c1-4e64-9f50-5f43dfd40a39 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:41.799856 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1337/2703] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:45.096756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1c2a3-0359-48eb-969c-111033af6a71 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:45.145588 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:45.160760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1c2a3-0359-48eb-969c-111033af6a71 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:45.165264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:45.166125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:45.166404 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:45.166676 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] PATH_INFO: `/v3/users/6d10372faa5e4f77b23ff9acce4578e7` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:45.168689 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d10372faa5e4f77b23ff9acce4578e7)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:45.170624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:45.216415 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d10372faa5e4f77b23ff9acce4578e7'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:45.217064 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d10372faa5e4f77b23ff9acce4578e7'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:45.224091 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d10372faa5e4f77b23ff9acce4578e7'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:45.236454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d10372faa5e4f77b23ff9acce4578e7'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:45.237117 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d10372faa5e4f77b23ff9acce4578e7'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:45.250062 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1367/2704] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:45 2022] DELETE /identity/v3/users/6d10372faa5e4f77b23ff9acce4578e7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:45.254576 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4460147-3b7d-4a70-81ba-26d1b56272d5 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:45.255052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4460147-3b7d-4a70-81ba-26d1b56272d5 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:45.255142 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4460147-3b7d-4a70-81ba-26d1b56272d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:45.272244 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-a4460147-3b7d-4a70-81ba-26d1b56272d5 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:45.273567 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a4460147-3b7d-4a70-81ba-26d1b56272d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:45.334310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1338/2705] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:45.344507 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:45.346206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:45.403862 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea4841-e902-4357-b321-393a8fe3a698 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:45.407486 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876906b-cf03-423d-9169-2d2177b4190c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:45.408116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876906b-cf03-423d-9169-2d2177b4190c service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:45.408418 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876906b-cf03-423d-9169-2d2177b4190c service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:45.408678 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876906b-cf03-423d-9169-2d2177b4190c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:45.417847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876906b-cf03-423d-9169-2d2177b4190c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:45.419194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876906b-cf03-423d-9169-2d2177b4190c service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:45.428661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1368/2706] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:45.794847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4460147-3b7d-4a70-81ba-26d1b56272d5 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:45.796825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-a4460147-3b7d-4a70-81ba-26d1b56272d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:45.810022 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4460147-3b7d-4a70-81ba-26d1b56272d5 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:45.814897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:45.815765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:45.816056 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:45.816334 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] PATH_INFO: `/v3/projects/60cf417b548a43d6817f075661cf131d` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:45.817575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60cf417b548a43d6817f075661cf131d)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:45.819485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:45.852737 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:45.871763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60cf417b548a43d6817f075661cf131d', 'request_id': 'req-405dbcb1-1962-4003-a768-59842dd13be3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:45.878033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60cf417b548a43d6817f075661cf131d', 'request_id': 'req-405dbcb1-1962-4003-a768-59842dd13be3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:45.886276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60cf417b548a43d6817f075661cf131d', 'request_id': 'req-405dbcb1-1962-4003-a768-59842dd13be3'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:45.899793 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] The token cache is being invalidate because project 60cf417b548a43d6817f075661cf131d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:45.900089 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:45.902121 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:45.905215 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1339/2707] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:45 2022] DELETE /identity/v3/projects/60cf417b548a43d6817f075661cf131d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:45.911644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876906b-cf03-423d-9169-2d2177b4190c service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:45.921029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876906b-cf03-423d-9169-2d2177b4190c service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:45.978620 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b305a5d-4229-44e2-834d-ca37b10862a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:45.980151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b305a5d-4229-44e2-834d-ca37b10862a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:45.980432 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b305a5d-4229-44e2-834d-ca37b10862a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:45.980708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b305a5d-4229-44e2-834d-ca37b10862a7 admin admin] PATH_INFO: `/v3/users/dcd0c068368443d0a1d35ac5e1ea642a` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:45.981941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b305a5d-4229-44e2-834d-ca37b10862a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcd0c068368443d0a1d35ac5e1ea642a)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:45.983849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b305a5d-4229-44e2-834d-ca37b10862a7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:45.996025 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:46.029955 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0b305a5d-4229-44e2-834d-ca37b10862a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcd0c068368443d0a1d35ac5e1ea642a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:46.030600 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0b305a5d-4229-44e2-834d-ca37b10862a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcd0c068368443d0a1d35ac5e1ea642a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:46.037544 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0b305a5d-4229-44e2-834d-ca37b10862a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcd0c068368443d0a1d35ac5e1ea642a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:46.053243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0b305a5d-4229-44e2-834d-ca37b10862a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcd0c068368443d0a1d35ac5e1ea642a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:46.054088 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0b305a5d-4229-44e2-834d-ca37b10862a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcd0c068368443d0a1d35ac5e1ea642a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:46.068971 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1369/2708] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:45 2022] DELETE /identity/v3/users/dcd0c068368443d0a1d35ac5e1ea642a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:46.365444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:46.367067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:46.381789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405dbcb1-1962-4003-a768-59842dd13be3 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:46.442102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813537d-c0c6-4cc5-bb26-851773a96042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:46.442880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813537d-c0c6-4cc5-bb26-851773a96042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:46.443163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813537d-c0c6-4cc5-bb26-851773a96042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:46.443435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813537d-c0c6-4cc5-bb26-851773a96042 admin admin] PATH_INFO: `/v3/projects/f25f197fdb4e4e7eb69ba7bc06a59c4b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:46.444616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813537d-c0c6-4cc5-bb26-851773a96042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f25f197fdb4e4e7eb69ba7bc06a59c4b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:46.446546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813537d-c0c6-4cc5-bb26-851773a96042 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:46.494033 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6813537d-c0c6-4cc5-bb26-851773a96042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f25f197fdb4e4e7eb69ba7bc06a59c4b', 'request_id': 'req-6813537d-c0c6-4cc5-bb26-851773a96042'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:46.500294 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6813537d-c0c6-4cc5-bb26-851773a96042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f25f197fdb4e4e7eb69ba7bc06a59c4b', 'request_id': 'req-6813537d-c0c6-4cc5-bb26-851773a96042'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:46.506935 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6813537d-c0c6-4cc5-bb26-851773a96042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f25f197fdb4e4e7eb69ba7bc06a59c4b', 'request_id': 'req-6813537d-c0c6-4cc5-bb26-851773a96042'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:46.518027 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6813537d-c0c6-4cc5-bb26-851773a96042 admin admin] The token cache is being invalidate because project f25f197fdb4e4e7eb69ba7bc06a59c4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:46.518312 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6813537d-c0c6-4cc5-bb26-851773a96042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:46.518996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6813537d-c0c6-4cc5-bb26-851773a96042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:46.520523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1340/2709] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:46 2022] DELETE /identity/v3/projects/f25f197fdb4e4e7eb69ba7bc06a59c4b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:46.525491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e4e42-e694-442e-99c1-dda20e3feb09 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:46.526005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e4e42-e694-442e-99c1-dda20e3feb09 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:46.526199 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e4e42-e694-442e-99c1-dda20e3feb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:46.545921 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-315e4e42-e694-442e-99c1-dda20e3feb09 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:46.547131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-315e4e42-e694-442e-99c1-dda20e3feb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:46.611020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1370/2710] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:46.618707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ebd60-ff2f-48ca-8aff-ef60dc777e42 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:46.618995 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ebd60-ff2f-48ca-8aff-ef60dc777e42 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:46.619272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ebd60-ff2f-48ca-8aff-ef60dc777e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:46.638137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9a9ebd60-ff2f-48ca-8aff-ef60dc777e42 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:46.639331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9a9ebd60-ff2f-48ca-8aff-ef60dc777e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:46.647204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1341/2711] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:12:46.651890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e4e42-e694-442e-99c1-dda20e3feb09 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:46.661412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e4e42-e694-442e-99c1-dda20e3feb09 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:46.665974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a33bd-9183-4c5e-9165-7bde27ca7947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:46.666619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a33bd-9183-4c5e-9165-7bde27ca7947 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:46.667441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a33bd-9183-4c5e-9165-7bde27ca7947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:46.667731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a33bd-9183-4c5e-9165-7bde27ca7947 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:46.668431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a33bd-9183-4c5e-9165-7bde27ca7947 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:46.668737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a33bd-9183-4c5e-9165-7bde27ca7947 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:46.670368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a33bd-9183-4c5e-9165-7bde27ca7947 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:46.681660 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1371/2712] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:46.685926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ebd60-ff2f-48ca-8aff-ef60dc777e42 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:46.695177 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ebd60-ff2f-48ca-8aff-ef60dc777e42 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:46.700095 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003592f-3901-4193-9b4a-12b64e709f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:46.700745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003592f-3901-4193-9b4a-12b64e709f24 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:46.701032 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003592f-3901-4193-9b4a-12b64e709f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:46.701374 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003592f-3901-4193-9b4a-12b64e709f24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:46.702071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003592f-3901-4193-9b4a-12b64e709f24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:46.703955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003592f-3901-4193-9b4a-12b64e709f24 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:46.729941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1342/2713] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:46 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:46.733962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a33bd-9183-4c5e-9165-7bde27ca7947 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:46.743708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a33bd-9183-4c5e-9165-7bde27ca7947 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:46.800857 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e1f71-dd08-422d-adcf-5fb6d52f4cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:46.801569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e1f71-dd08-422d-adcf-5fb6d52f4cfe admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:46.801910 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e1f71-dd08-422d-adcf-5fb6d52f4cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:46.802138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e1f71-dd08-422d-adcf-5fb6d52f4cfe admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:46.802845 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e1f71-dd08-422d-adcf-5fb6d52f4cfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:46.804745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e1f71-dd08-422d-adcf-5fb6d52f4cfe admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:46.827106 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1372/2714] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:46 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:46.831177 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003592f-3901-4193-9b4a-12b64e709f24 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:46.842230 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003592f-3901-4193-9b4a-12b64e709f24 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:46.847130 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfeaf8-9997-4544-8fbe-4d1befabed15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:46.847789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfeaf8-9997-4544-8fbe-4d1befabed15 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:46.848076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfeaf8-9997-4544-8fbe-4d1befabed15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:46.848354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfeaf8-9997-4544-8fbe-4d1befabed15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:46.849154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfeaf8-9997-4544-8fbe-4d1befabed15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:46.850751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfeaf8-9997-4544-8fbe-4d1befabed15 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:46.861948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1343/2715] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:46.865933 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e1f71-dd08-422d-adcf-5fb6d52f4cfe admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:46.876411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e1f71-dd08-422d-adcf-5fb6d52f4cfe admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:46.881372 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35387d30-1fcd-4dbc-ac14-e2c9de86696e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:46.882069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35387d30-1fcd-4dbc-ac14-e2c9de86696e admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:46.882352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35387d30-1fcd-4dbc-ac14-e2c9de86696e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:46.882622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35387d30-1fcd-4dbc-ac14-e2c9de86696e admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:46.883346 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35387d30-1fcd-4dbc-ac14-e2c9de86696e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:46.885065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35387d30-1fcd-4dbc-ac14-e2c9de86696e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:46.887137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:46.898930 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1373/2716] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:46.903230 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfeaf8-9997-4544-8fbe-4d1befabed15 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:46.909858 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:46.914857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfeaf8-9997-4544-8fbe-4d1befabed15 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:46.921569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a5e3b-9515-4450-82ad-05ab7ff33fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:46.923088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a5e3b-9515-4450-82ad-05ab7ff33fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:46.923088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a5e3b-9515-4450-82ad-05ab7ff33fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:46.923635 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a5e3b-9515-4450-82ad-05ab7ff33fbe admin admin] PATH_INFO: `/v3/projects/3e7bd13b11c64a9e95e37d150edd14fe/users/37dad6393b304a29bce0ce9415809a68/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:46.939028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a5e3b-9515-4450-82ad-05ab7ff33fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e7bd13b11c64a9e95e37d150edd14fe, user_id=37dad6393b304a29bce0ce9415809a68, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:46.941860 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a5e3b-9515-4450-82ad-05ab7ff33fbe admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:46.955986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1344/2717] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:46 2022] PUT /identity/v3/projects/3e7bd13b11c64a9e95e37d150edd14fe/users/37dad6393b304a29bce0ce9415809a68/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:46.966816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe9b3-c5b8-4694-b84c-23c47d72b592 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:46.967104 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe9b3-c5b8-4694-b84c-23c47d72b592 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:46.967376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe9b3-c5b8-4694-b84c-23c47d72b592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:46.992856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-6bbbe9b3-c5b8-4694-b84c-23c47d72b592 None None] MFA Rules not processed for user `37dad6393b304a29bce0ce9415809a68`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:46.994080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6bbbe9b3-c5b8-4694-b84c-23c47d72b592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:47.059628 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1374/2718] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:47.064086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a5e3b-9515-4450-82ad-05ab7ff33fbe admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:47.128838 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a5e3b-9515-4450-82ad-05ab7ff33fbe admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:47.133736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669507fa-d175-4518-ac47-e8d0e568952b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:47.134426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669507fa-d175-4518-ac47-e8d0e568952b admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:47.134712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669507fa-d175-4518-ac47-e8d0e568952b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:47.134988 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669507fa-d175-4518-ac47-e8d0e568952b admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:47.136361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669507fa-d175-4518-ac47-e8d0e568952b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:47.138434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669507fa-d175-4518-ac47-e8d0e568952b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:47.178063 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1345/2719] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:47 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:47.179284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbe9b3-c5b8-4694-b84c-23c47d72b592 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:47.190876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbe9b3-c5b8-4694-b84c-23c47d72b592 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:47.253198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f9b80-3654-4e17-89a0-de4decbd297d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:47.253804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f9b80-3654-4e17-89a0-de4decbd297d admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:47.254092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f9b80-3654-4e17-89a0-de4decbd297d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:47.254370 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f9b80-3654-4e17-89a0-de4decbd297d admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:47.255077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f9b80-3654-4e17-89a0-de4decbd297d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:47.257175 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f9b80-3654-4e17-89a0-de4decbd297d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:47.279474 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1375/2720] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:47 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:12:47.283596 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669507fa-d175-4518-ac47-e8d0e568952b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:47.294148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669507fa-d175-4518-ac47-e8d0e568952b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:47.300129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a0ee2-6657-4719-96c0-eb11c6f18f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:47.300896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a0ee2-6657-4719-96c0-eb11c6f18f42 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:47.301296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a0ee2-6657-4719-96c0-eb11c6f18f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:47.301637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a0ee2-6657-4719-96c0-eb11c6f18f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:47.302418 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a0ee2-6657-4719-96c0-eb11c6f18f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:47.304231 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a0ee2-6657-4719-96c0-eb11c6f18f42 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:47.315288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1346/2721] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:12:47.319134 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f9b80-3654-4e17-89a0-de4decbd297d admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:47.330034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f9b80-3654-4e17-89a0-de4decbd297d admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:47.334863 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630fc53-b477-4632-9e53-9e516085df02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:47.335519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630fc53-b477-4632-9e53-9e516085df02 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:47.335805 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630fc53-b477-4632-9e53-9e516085df02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:47.336084 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630fc53-b477-4632-9e53-9e516085df02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:47.336817 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630fc53-b477-4632-9e53-9e516085df02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:47.338453 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630fc53-b477-4632-9e53-9e516085df02 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:47.349535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1376/2722] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:47.353449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a0ee2-6657-4719-96c0-eb11c6f18f42 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:47.363359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a0ee2-6657-4719-96c0-eb11c6f18f42 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:47.367967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b85e9b-a70d-402e-a37d-60f6eef71aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:47.368847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b85e9b-a70d-402e-a37d-60f6eef71aa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:47.369167 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b85e9b-a70d-402e-a37d-60f6eef71aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:47.369449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b85e9b-a70d-402e-a37d-60f6eef71aa5 admin admin] PATH_INFO: `/v3/projects/75378a4b5eea438fb64976b13911fce7/users/bf46d96cecf548ad8f68146f0b786b93/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:47.383684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b85e9b-a70d-402e-a37d-60f6eef71aa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75378a4b5eea438fb64976b13911fce7, user_id=bf46d96cecf548ad8f68146f0b786b93, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:47.386900 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b85e9b-a70d-402e-a37d-60f6eef71aa5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:47.399862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1347/2723] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:47 2022] PUT /identity/v3/projects/75378a4b5eea438fb64976b13911fce7/users/bf46d96cecf548ad8f68146f0b786b93/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:47.410692 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c0287-828c-4615-85cb-33b28cd7f47d None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:47.410983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c0287-828c-4615-85cb-33b28cd7f47d None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:47.411286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c0287-828c-4615-85cb-33b28cd7f47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:47.440060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-222c0287-828c-4615-85cb-33b28cd7f47d None None] MFA Rules not processed for user `bf46d96cecf548ad8f68146f0b786b93`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:47.441415 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-222c0287-828c-4615-85cb-33b28cd7f47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:47.496473 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:47.508273 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1377/2724] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:47.513463 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b85e9b-a70d-402e-a37d-60f6eef71aa5 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:47.577818 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b85e9b-a70d-402e-a37d-60f6eef71aa5 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:47.582498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208573f8-9121-40d0-b869-ce6b0b6d4fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:47.583152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208573f8-9121-40d0-b869-ce6b0b6d4fbf admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:47.583441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208573f8-9121-40d0-b869-ce6b0b6d4fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:47.583720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208573f8-9121-40d0-b869-ce6b0b6d4fbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:47.584404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208573f8-9121-40d0-b869-ce6b0b6d4fbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:47.586316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208573f8-9121-40d0-b869-ce6b0b6d4fbf admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:47.611744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1348/2725] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:47 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:47.615839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c0287-828c-4615-85cb-33b28cd7f47d None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:47.626854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c0287-828c-4615-85cb-33b28cd7f47d None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:47.684089 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d8430-2622-4f43-97da-a582d628074c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:47.684764 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8430-2622-4f43-97da-a582d628074c admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:47.685049 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8430-2622-4f43-97da-a582d628074c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:47.685360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8430-2622-4f43-97da-a582d628074c admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:47.686060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d8430-2622-4f43-97da-a582d628074c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:47.687979 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d8430-2622-4f43-97da-a582d628074c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:47.711395 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1378/2726] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:47 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:47.712572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208573f8-9121-40d0-b869-ce6b0b6d4fbf admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:47.723379 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208573f8-9121-40d0-b869-ce6b0b6d4fbf admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:47.728321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1699c6-2d82-4bd3-9809-e59fb34e2b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:47.729232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1699c6-2d82-4bd3-9809-e59fb34e2b39 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:47.729660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1699c6-2d82-4bd3-9809-e59fb34e2b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:47.729987 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1699c6-2d82-4bd3-9809-e59fb34e2b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:47.730833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1699c6-2d82-4bd3-9809-e59fb34e2b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:47.732914 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1699c6-2d82-4bd3-9809-e59fb34e2b39 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:47.745716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1349/2727] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:47.749816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d8430-2622-4f43-97da-a582d628074c admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:47.760039 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d8430-2622-4f43-97da-a582d628074c admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:47.765974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90e800-9eee-4644-b9e3-01d4df8846ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:47.766876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90e800-9eee-4644-b9e3-01d4df8846ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:47.767287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90e800-9eee-4644-b9e3-01d4df8846ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:47.767572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90e800-9eee-4644-b9e3-01d4df8846ab admin admin] PATH_INFO: `/v3/projects/8fe2304120544c43b6191bc3ede37be3/users/218b529cb6a8478484d9f961e35aa937/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:47.783099 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90e800-9eee-4644-b9e3-01d4df8846ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe2304120544c43b6191bc3ede37be3, user_id=218b529cb6a8478484d9f961e35aa937, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:47.785824 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90e800-9eee-4644-b9e3-01d4df8846ab admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:47.799869 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1379/2728] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:47 2022] PUT /identity/v3/projects/8fe2304120544c43b6191bc3ede37be3/users/218b529cb6a8478484d9f961e35aa937/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:47.804088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1699c6-2d82-4bd3-9809-e59fb34e2b39 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:47.811751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:47.816107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1699c6-2d82-4bd3-9809-e59fb34e2b39 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:47.881783 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed40490-49d8-4f52-be1c-9de2039ecc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:47.882463 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed40490-49d8-4f52-be1c-9de2039ecc79 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:47.882751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed40490-49d8-4f52-be1c-9de2039ecc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:47.883034 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed40490-49d8-4f52-be1c-9de2039ecc79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:47.883938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed40490-49d8-4f52-be1c-9de2039ecc79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:47.885613 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed40490-49d8-4f52-be1c-9de2039ecc79 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:47.903449 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1350/2729] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:47.908190 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90e800-9eee-4644-b9e3-01d4df8846ab admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:47.933953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90e800-9eee-4644-b9e3-01d4df8846ab admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:47.939151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ae8ab-feae-44d7-836c-559a6730ff21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:47.940026 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae8ab-feae-44d7-836c-559a6730ff21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:47.940309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae8ab-feae-44d7-836c-559a6730ff21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:47.940582 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae8ab-feae-44d7-836c-559a6730ff21 admin admin] PATH_INFO: `/v3/projects/8fe2304120544c43b6191bc3ede37be3/users/218b529cb6a8478484d9f961e35aa937/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:47.942874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ae8ab-feae-44d7-836c-559a6730ff21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe2304120544c43b6191bc3ede37be3, user_id=218b529cb6a8478484d9f961e35aa937, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:47.945606 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ae8ab-feae-44d7-836c-559a6730ff21 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:47.956419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1380/2730] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:47 2022] PUT /identity/v3/projects/8fe2304120544c43b6191bc3ede37be3/users/218b529cb6a8478484d9f961e35aa937/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:47.967647 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e546119-8f6d-4ecf-a757-ca393dd2070d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:47.968019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e546119-8f6d-4ecf-a757-ca393dd2070d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:47.968368 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e546119-8f6d-4ecf-a757-ca393dd2070d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:48.000416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5e546119-8f6d-4ecf-a757-ca393dd2070d None None] MFA Rules not processed for user `218b529cb6a8478484d9f961e35aa937`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:48.001829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5e546119-8f6d-4ecf-a757-ca393dd2070d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:48.093201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1351/2731] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:48.105830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ae8ab-feae-44d7-836c-559a6730ff21 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:48.107907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a85ae8ab-feae-44d7-836c-559a6730ff21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:48.176507 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ae8ab-feae-44d7-836c-559a6730ff21 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:48.180158 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9eda34-7cb5-4b16-8e1a-077b4c0b8e40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:48.180799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eda34-7cb5-4b16-8e1a-077b4c0b8e40 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:48.181103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eda34-7cb5-4b16-8e1a-077b4c0b8e40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:48.181388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eda34-7cb5-4b16-8e1a-077b4c0b8e40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:48.190858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9eda34-7cb5-4b16-8e1a-077b4c0b8e40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:48.192286 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9eda34-7cb5-4b16-8e1a-077b4c0b8e40 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:48.202376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1381/2732] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:48.240268 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e546119-8f6d-4ecf-a757-ca393dd2070d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:48.241703 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5e546119-8f6d-4ecf-a757-ca393dd2070d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:48.254083 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e546119-8f6d-4ecf-a757-ca393dd2070d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:48.257799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bf443-1408-44db-9ac2-33b7cdecf17d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:48.258464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bf443-1408-44db-9ac2-33b7cdecf17d service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:48.258755 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bf443-1408-44db-9ac2-33b7cdecf17d service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:48.259038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bf443-1408-44db-9ac2-33b7cdecf17d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:48.315156 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bf443-1408-44db-9ac2-33b7cdecf17d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:48.316506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bf443-1408-44db-9ac2-33b7cdecf17d service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:48.326357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1352/2733] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:49.500221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9eda34-7cb5-4b16-8e1a-077b4c0b8e40 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:49.501733 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ee9eda34-7cb5-4b16-8e1a-077b4c0b8e40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:49.555737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:49.560331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9eda34-7cb5-4b16-8e1a-077b4c0b8e40 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:49.564357 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6df556-38d2-422f-896f-b09a34131f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:49.565000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6df556-38d2-422f-896f-b09a34131f49 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:49.565329 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6df556-38d2-422f-896f-b09a34131f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:49.565607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6df556-38d2-422f-896f-b09a34131f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:49.567365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-7c6df556-38d2-422f-896f-b09a34131f49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:49.629656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6df556-38d2-422f-896f-b09a34131f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:49.631089 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6df556-38d2-422f-896f-b09a34131f49 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:49.640804 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1382/2734] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:51.221971 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bf443-1408-44db-9ac2-33b7cdecf17d service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:51.249213 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=100 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:51.287110 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bf443-1408-44db-9ac2-33b7cdecf17d service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:51.291832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:51.292990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:51.293304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:51.293583 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] PATH_INFO: `/v3/users/37dad6393b304a29bce0ce9415809a68` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:51.294825 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=37dad6393b304a29bce0ce9415809a68)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:51.296795 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:51.344216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37dad6393b304a29bce0ce9415809a68'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:51.344872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37dad6393b304a29bce0ce9415809a68'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:51.352415 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37dad6393b304a29bce0ce9415809a68'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:51.353035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37dad6393b304a29bce0ce9415809a68'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:51.365780 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37dad6393b304a29bce0ce9415809a68'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:51.380162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1353/2735] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:51 2022] DELETE /identity/v3/users/37dad6393b304a29bce0ce9415809a68 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:51.384663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca3b89-a5d2-4139-8524-a4265558b142 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:51.385186 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca3b89-a5d2-4139-8524-a4265558b142 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:51.385662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca3b89-a5d2-4139-8524-a4265558b142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:51.403434 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-85ca3b89-a5d2-4139-8524-a4265558b142 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:51.404754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-85ca3b89-a5d2-4139-8524-a4265558b142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:51.467014 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1383/2736] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:51.476782 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:51.478210 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:51.560949 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559de975-1382-4431-93b8-df468aa327cb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:51.575238 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e666a31-9758-4855-ab66-ec366d3f409a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:51.576147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e666a31-9758-4855-ab66-ec366d3f409a service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:51.576529 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e666a31-9758-4855-ab66-ec366d3f409a service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:51.576906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e666a31-9758-4855-ab66-ec366d3f409a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:51.587267 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e666a31-9758-4855-ab66-ec366d3f409a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:51.589043 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e666a31-9758-4855-ab66-ec366d3f409a service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:51.603720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1354/2737] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:52.190279 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca3b89-a5d2-4139-8524-a4265558b142 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:52.191911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-85ca3b89-a5d2-4139-8524-a4265558b142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:52.207045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca3b89-a5d2-4139-8524-a4265558b142 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:52.212342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:52.214046 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:52.214319 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:52.214704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] PATH_INFO: `/v3/projects/3e7bd13b11c64a9e95e37d150edd14fe` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:52.216798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e7bd13b11c64a9e95e37d150edd14fe)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:52.219491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:52.280397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e7bd13b11c64a9e95e37d150edd14fe', 'request_id': 'req-0c980d23-0e3a-4be2-968b-0b481491ab05'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.287270 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e7bd13b11c64a9e95e37d150edd14fe', 'request_id': 'req-0c980d23-0e3a-4be2-968b-0b481491ab05'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.296725 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e7bd13b11c64a9e95e37d150edd14fe', 'request_id': 'req-0c980d23-0e3a-4be2-968b-0b481491ab05'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.311653 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] The token cache is being invalidate because project 3e7bd13b11c64a9e95e37d150edd14fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:52.312041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.312783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.317494 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1384/2738] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:52 2022] DELETE /identity/v3/projects/3e7bd13b11c64a9e95e37d150edd14fe => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:52.318770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e666a31-9758-4855-ab66-ec366d3f409a service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:52.329445 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e666a31-9758-4855-ab66-ec366d3f409a service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:52.376030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:52.390906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:52.391786 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:52.392088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:52.392346 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] PATH_INFO: `/v3/users/bf46d96cecf548ad8f68146f0b786b93` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:52.393553 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf46d96cecf548ad8f68146f0b786b93)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:52.395464 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:52.445791 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf46d96cecf548ad8f68146f0b786b93'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.448924 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf46d96cecf548ad8f68146f0b786b93'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.458731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf46d96cecf548ad8f68146f0b786b93'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.459356 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf46d96cecf548ad8f68146f0b786b93'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.473165 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf46d96cecf548ad8f68146f0b786b93'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.486720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1355/2739] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:52 2022] DELETE /identity/v3/users/bf46d96cecf548ad8f68146f0b786b93 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:12:52.781521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:52.782986 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:52.801451 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c980d23-0e3a-4be2-968b-0b481491ab05 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:52.822007 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:52.874704 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:52.875574 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:52.875865 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:52.876147 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] PATH_INFO: `/v3/projects/75378a4b5eea438fb64976b13911fce7` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:52.877357 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] RBAC: Authorizing `identity:delete_project(project_id=75378a4b5eea438fb64976b13911fce7)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:52.879255 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:52.925326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75378a4b5eea438fb64976b13911fce7', 'request_id': 'req-95f6e437-477d-409e-a985-2be70ef61bac'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.931377 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75378a4b5eea438fb64976b13911fce7', 'request_id': 'req-95f6e437-477d-409e-a985-2be70ef61bac'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.940118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75378a4b5eea438fb64976b13911fce7', 'request_id': 'req-95f6e437-477d-409e-a985-2be70ef61bac'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.951291 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] The token cache is being invalidate because project 75378a4b5eea438fb64976b13911fce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:52.951520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.952167 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:52.953749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1385/2740] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:52 2022] DELETE /identity/v3/projects/75378a4b5eea438fb64976b13911fce7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:52.957807 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:52.959212 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:52.971957 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6260c3b-a874-46e3-8b57-3ee83ece2877 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:53.027603 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344615c-d673-44c1-83a5-6c1b138454e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:53.028471 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344615c-d673-44c1-83a5-6c1b138454e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:53.028757 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344615c-d673-44c1-83a5-6c1b138454e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:53.029039 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344615c-d673-44c1-83a5-6c1b138454e5 admin admin] PATH_INFO: `/v3/users/218b529cb6a8478484d9f961e35aa937` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:53.030252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344615c-d673-44c1-83a5-6c1b138454e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=218b529cb6a8478484d9f961e35aa937)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:53.032151 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344615c-d673-44c1-83a5-6c1b138454e5 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:53.078416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0344615c-d673-44c1-83a5-6c1b138454e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218b529cb6a8478484d9f961e35aa937'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:53.079051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0344615c-d673-44c1-83a5-6c1b138454e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218b529cb6a8478484d9f961e35aa937'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:53.086240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0344615c-d673-44c1-83a5-6c1b138454e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218b529cb6a8478484d9f961e35aa937'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:53.086853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0344615c-d673-44c1-83a5-6c1b138454e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218b529cb6a8478484d9f961e35aa937'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:53.098501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:53.101075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0344615c-d673-44c1-83a5-6c1b138454e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218b529cb6a8478484d9f961e35aa937'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:53.115073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1356/2741] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:52 2022] DELETE /identity/v3/users/218b529cb6a8478484d9f961e35aa937 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:53.386936 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:53.388388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:53.401730 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6e437-477d-409e-a985-2be70ef61bac admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:53.461060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc401ec-3c67-4752-879d-ae605734c9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:53.461881 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc401ec-3c67-4752-879d-ae605734c9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:53.462168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc401ec-3c67-4752-879d-ae605734c9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:53.462449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc401ec-3c67-4752-879d-ae605734c9ef admin admin] PATH_INFO: `/v3/projects/8fe2304120544c43b6191bc3ede37be3` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:53.463866 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc401ec-3c67-4752-879d-ae605734c9ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fe2304120544c43b6191bc3ede37be3)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:53.465818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc401ec-3c67-4752-879d-ae605734c9ef admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:53.513765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bdc401ec-3c67-4752-879d-ae605734c9ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fe2304120544c43b6191bc3ede37be3', 'request_id': 'req-bdc401ec-3c67-4752-879d-ae605734c9ef'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:53.519726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bdc401ec-3c67-4752-879d-ae605734c9ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fe2304120544c43b6191bc3ede37be3', 'request_id': 'req-bdc401ec-3c67-4752-879d-ae605734c9ef'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:53.521380 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:53.527841 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bdc401ec-3c67-4752-879d-ae605734c9ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fe2304120544c43b6191bc3ede37be3', 'request_id': 'req-bdc401ec-3c67-4752-879d-ae605734c9ef'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:53.539526 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bdc401ec-3c67-4752-879d-ae605734c9ef admin admin] The token cache is being invalidate because project 8fe2304120544c43b6191bc3ede37be3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80229) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:53.539818 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bdc401ec-3c67-4752-879d-ae605734c9ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:53.540450 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-bdc401ec-3c67-4752-879d-ae605734c9ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:53.542116 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1386/2742] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:53 2022] DELETE /identity/v3/projects/8fe2304120544c43b6191bc3ede37be3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:12:53.548121 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bd2ab-801f-4a96-a204-bbedb7cee5f4 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:53.548645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bd2ab-801f-4a96-a204-bbedb7cee5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:53.548736 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bd2ab-801f-4a96-a204-bbedb7cee5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:53.567296 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b21bd2ab-801f-4a96-a204-bbedb7cee5f4 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:53.568493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b21bd2ab-801f-4a96-a204-bbedb7cee5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:53.627444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1357/2743] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:53.636222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a89b5-b3d0-4e18-81a0-e1e4753689bc None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:53.636404 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a89b5-b3d0-4e18-81a0-e1e4753689bc None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:53.636670 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a89b5-b3d0-4e18-81a0-e1e4753689bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:53.654990 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-225a89b5-b3d0-4e18-81a0-e1e4753689bc None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:53.656280 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-225a89b5-b3d0-4e18-81a0-e1e4753689bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:53.665644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1387/2744] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:53.669942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bd2ab-801f-4a96-a204-bbedb7cee5f4 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:53.679833 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bd2ab-801f-4a96-a204-bbedb7cee5f4 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:53.685227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ffb28-4fda-4d64-bba7-7f24afe85f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:53.685951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ffb28-4fda-4d64-bba7-7f24afe85f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:53.686243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ffb28-4fda-4d64-bba7-7f24afe85f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:53.686523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ffb28-4fda-4d64-bba7-7f24afe85f81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:53.687243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ffb28-4fda-4d64-bba7-7f24afe85f81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80228) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:53.687560 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ffb28-4fda-4d64-bba7-7f24afe85f81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:53.689227 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ffb28-4fda-4d64-bba7-7f24afe85f81 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:53.701578 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1358/2745] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:53.705909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a89b5-b3d0-4e18-81a0-e1e4753689bc None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:53.715368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a89b5-b3d0-4e18-81a0-e1e4753689bc None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:53.720076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a772-6417-4cc9-93f3-f0b898fbbebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:53.720976 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a772-6417-4cc9-93f3-f0b898fbbebd admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:53.721291 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a772-6417-4cc9-93f3-f0b898fbbebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:53.721571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a772-6417-4cc9-93f3-f0b898fbbebd admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:53.722266 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922a772-6417-4cc9-93f3-f0b898fbbebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:53.724165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922a772-6417-4cc9-93f3-f0b898fbbebd admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:53.749529 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1388/2746] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:53 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:53.753572 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ffb28-4fda-4d64-bba7-7f24afe85f81 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:53.764743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ffb28-4fda-4d64-bba7-7f24afe85f81 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:53.824199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b3c9d-d8f8-4000-aa6d-71a9b23ad85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:53.824874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b3c9d-d8f8-4000-aa6d-71a9b23ad85f admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:53.825192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b3c9d-d8f8-4000-aa6d-71a9b23ad85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:53.825472 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b3c9d-d8f8-4000-aa6d-71a9b23ad85f admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:53.826170 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b3c9d-d8f8-4000-aa6d-71a9b23ad85f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:53.828067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b3c9d-d8f8-4000-aa6d-71a9b23ad85f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:53.847882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1359/2747] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:53 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:53.852037 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a772-6417-4cc9-93f3-f0b898fbbebd admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:53.862992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a772-6417-4cc9-93f3-f0b898fbbebd admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:53.867623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d9be2-d4a4-46a3-a0a9-078e91c7ff5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:53.868274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d9be2-d4a4-46a3-a0a9-078e91c7ff5b admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:53.868563 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d9be2-d4a4-46a3-a0a9-078e91c7ff5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:53.868842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d9be2-d4a4-46a3-a0a9-078e91c7ff5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:53.869772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d9be2-d4a4-46a3-a0a9-078e91c7ff5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:53.871952 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d9be2-d4a4-46a3-a0a9-078e91c7ff5b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:53.882207 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1389/2748] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:53.886136 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b3c9d-d8f8-4000-aa6d-71a9b23ad85f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:53.896397 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b3c9d-d8f8-4000-aa6d-71a9b23ad85f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:53.901075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65217728-1a49-484f-b132-90d2c9ef1f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:53.901769 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65217728-1a49-484f-b132-90d2c9ef1f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:53.902050 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65217728-1a49-484f-b132-90d2c9ef1f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:53.902324 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65217728-1a49-484f-b132-90d2c9ef1f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:53.903078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65217728-1a49-484f-b132-90d2c9ef1f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:53.904644 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65217728-1a49-484f-b132-90d2c9ef1f1e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:53.915043 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1360/2749] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:53.919906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d9be2-d4a4-46a3-a0a9-078e91c7ff5b admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:53.930139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d9be2-d4a4-46a3-a0a9-078e91c7ff5b admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:53.935811 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bb775-46c7-4626-8334-83edc4560679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:53.936694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bb775-46c7-4626-8334-83edc4560679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:53.936981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bb775-46c7-4626-8334-83edc4560679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:53.937290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bb775-46c7-4626-8334-83edc4560679 admin admin] PATH_INFO: `/v3/projects/c1ba3e44a1dc4b4b828a173e9471bec9/users/c1331c3bfca64ad68a1a500150e9f4d0/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:53.953306 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bb775-46c7-4626-8334-83edc4560679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ba3e44a1dc4b4b828a173e9471bec9, user_id=c1331c3bfca64ad68a1a500150e9f4d0, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:53.956053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bb775-46c7-4626-8334-83edc4560679 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:53.968933 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1390/2750] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:53 2022] PUT /identity/v3/projects/c1ba3e44a1dc4b4b828a173e9471bec9/users/c1331c3bfca64ad68a1a500150e9f4d0/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:53.980935 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8c125-49e9-4626-aa26-0d8fff111093 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:53.981329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8c125-49e9-4626-aa26-0d8fff111093 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:53.981612 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8c125-49e9-4626-aa26-0d8fff111093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:54.007811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-16f8c125-49e9-4626-aa26-0d8fff111093 None None] MFA Rules not processed for user `c1331c3bfca64ad68a1a500150e9f4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:54.009013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-16f8c125-49e9-4626-aa26-0d8fff111093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:54.048281 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:54.073719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1361/2751] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:54.078147 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bb775-46c7-4626-8334-83edc4560679 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:54.138732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bb775-46c7-4626-8334-83edc4560679 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:54.143345 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fffbe7-d1a7-4dfa-91b1-d26a06fcb783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:54.143993 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fffbe7-d1a7-4dfa-91b1-d26a06fcb783 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:54.144278 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fffbe7-d1a7-4dfa-91b1-d26a06fcb783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:54.144559 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fffbe7-d1a7-4dfa-91b1-d26a06fcb783 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:54.145287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fffbe7-d1a7-4dfa-91b1-d26a06fcb783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:54.147184 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fffbe7-d1a7-4dfa-91b1-d26a06fcb783 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:54.176001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1391/2752] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:54 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:54.177223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8c125-49e9-4626-aa26-0d8fff111093 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:54.188005 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8c125-49e9-4626-aa26-0d8fff111093 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:54.244355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e179994-6c66-48ac-8b82-7ec0febb5005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:54.245137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e179994-6c66-48ac-8b82-7ec0febb5005 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:54.245459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e179994-6c66-48ac-8b82-7ec0febb5005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:54.245743 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e179994-6c66-48ac-8b82-7ec0febb5005 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:54.246427 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e179994-6c66-48ac-8b82-7ec0febb5005 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:54.248409 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e179994-6c66-48ac-8b82-7ec0febb5005 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:54.270088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1362/2753] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:54 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:54.271595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fffbe7-d1a7-4dfa-91b1-d26a06fcb783 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:54.282479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fffbe7-d1a7-4dfa-91b1-d26a06fcb783 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:54.287085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d53e94-2ddb-4155-87c6-45c85e57bd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:54.287735 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d53e94-2ddb-4155-87c6-45c85e57bd36 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:54.288022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d53e94-2ddb-4155-87c6-45c85e57bd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:54.288324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d53e94-2ddb-4155-87c6-45c85e57bd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:54.289044 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d53e94-2ddb-4155-87c6-45c85e57bd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:54.290680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d53e94-2ddb-4155-87c6-45c85e57bd36 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:54.302566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1392/2754] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:54.306694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e179994-6c66-48ac-8b82-7ec0febb5005 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:54.317103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e179994-6c66-48ac-8b82-7ec0febb5005 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:54.321720 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8acab-a796-45d4-91a6-4a17f75c095c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:54.322589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8acab-a796-45d4-91a6-4a17f75c095c admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:54.322935 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8acab-a796-45d4-91a6-4a17f75c095c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:54.323283 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8acab-a796-45d4-91a6-4a17f75c095c admin admin] PATH_INFO: `/v3/projects/017e18d18da94630b262454378906e7a/users/bbbefd227437470dbdd5b2861b2b152c/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:54.337853 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8acab-a796-45d4-91a6-4a17f75c095c admin admin] RBAC: Authorizing `identity:create_grant(project_id=017e18d18da94630b262454378906e7a, user_id=bbbefd227437470dbdd5b2861b2b152c, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:54.341027 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8acab-a796-45d4-91a6-4a17f75c095c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:54.353446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1363/2755] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:54 2022] PUT /identity/v3/projects/017e18d18da94630b262454378906e7a/users/bbbefd227437470dbdd5b2861b2b152c/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:54.358193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d53e94-2ddb-4155-87c6-45c85e57bd36 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:54.368139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d53e94-2ddb-4155-87c6-45c85e57bd36 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:54.414448 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:54.424800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a368ccc-0ed8-49a7-9fe2-977229e1e49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:54.425525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a368ccc-0ed8-49a7-9fe2-977229e1e49c admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:54.425854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a368ccc-0ed8-49a7-9fe2-977229e1e49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:54.426302 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a368ccc-0ed8-49a7-9fe2-977229e1e49c admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:54.427055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a368ccc-0ed8-49a7-9fe2-977229e1e49c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:54.428633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a368ccc-0ed8-49a7-9fe2-977229e1e49c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:54.438916 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1393/2756] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:12:54.442962 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8acab-a796-45d4-91a6-4a17f75c095c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:54.453887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8acab-a796-45d4-91a6-4a17f75c095c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:54.458478 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2a621-e15a-4b14-bc50-495fb3b538b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:54.459383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2a621-e15a-4b14-bc50-495fb3b538b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:54.459665 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2a621-e15a-4b14-bc50-495fb3b538b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:54.459941 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2a621-e15a-4b14-bc50-495fb3b538b1 admin admin] PATH_INFO: `/v3/projects/017e18d18da94630b262454378906e7a/users/bbbefd227437470dbdd5b2861b2b152c/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:54.461919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2a621-e15a-4b14-bc50-495fb3b538b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=017e18d18da94630b262454378906e7a, user_id=bbbefd227437470dbdd5b2861b2b152c, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:54.464745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2a621-e15a-4b14-bc50-495fb3b538b1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:54.474984 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1364/2757] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:54 2022] PUT /identity/v3/projects/017e18d18da94630b262454378906e7a/users/bbbefd227437470dbdd5b2861b2b152c/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:54.485782 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0dc21-5261-4646-a012-ef679517da87 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:54.486080 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0dc21-5261-4646-a012-ef679517da87 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:54.486364 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0dc21-5261-4646-a012-ef679517da87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:54.513558 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-f5d0dc21-5261-4646-a012-ef679517da87 None None] MFA Rules not processed for user `bbbefd227437470dbdd5b2861b2b152c`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:54.514708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f5d0dc21-5261-4646-a012-ef679517da87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:54.582811 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1394/2758] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:54 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:54.594329 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2a621-e15a-4b14-bc50-495fb3b538b1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:54.596535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dfe2a621-e15a-4b14-bc50-495fb3b538b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:54.653381 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2a621-e15a-4b14-bc50-495fb3b538b1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:54.657064 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695f721-14b6-45cd-99d9-060ed3f1328b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:54.657735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695f721-14b6-45cd-99d9-060ed3f1328b service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:54.658018 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695f721-14b6-45cd-99d9-060ed3f1328b service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:54.658295 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695f721-14b6-45cd-99d9-060ed3f1328b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:54.667820 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695f721-14b6-45cd-99d9-060ed3f1328b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:54.669279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695f721-14b6-45cd-99d9-060ed3f1328b service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:54.678616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1365/2759] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:54 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:54.758488 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0dc21-5261-4646-a012-ef679517da87 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:54.759954 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f5d0dc21-5261-4646-a012-ef679517da87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:54.772222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0dc21-5261-4646-a012-ef679517da87 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:54.777178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484142e8-fd3e-4728-a1cc-16163271caea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:54.777178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484142e8-fd3e-4728-a1cc-16163271caea service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:54.777178 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484142e8-fd3e-4728-a1cc-16163271caea service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:54.781892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484142e8-fd3e-4728-a1cc-16163271caea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:54.827671 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695f721-14b6-45cd-99d9-060ed3f1328b service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:54.829103 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1695f721-14b6-45cd-99d9-060ed3f1328b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:54.839014 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484142e8-fd3e-4728-a1cc-16163271caea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:54.840974 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484142e8-fd3e-4728-a1cc-16163271caea service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:54.850290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1395/2760] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:54.872824 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:54.896620 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695f721-14b6-45cd-99d9-060ed3f1328b service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:54.901751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f062c-087c-4b5d-87c5-17a023e16116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:54.902513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f062c-087c-4b5d-87c5-17a023e16116 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:54.903010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f062c-087c-4b5d-87c5-17a023e16116 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:54.903380 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f062c-087c-4b5d-87c5-17a023e16116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:54.905200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ec4f062c-087c-4b5d-87c5-17a023e16116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:54.971716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f062c-087c-4b5d-87c5-17a023e16116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:54.973383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f062c-087c-4b5d-87c5-17a023e16116 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:54.984203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1366/2761] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:12:57.993489 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484142e8-fd3e-4728-a1cc-16163271caea service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:57.995717 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-484142e8-fd3e-4728-a1cc-16163271caea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:58.018177 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484142e8-fd3e-4728-a1cc-16163271caea service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:58.023527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47969e7e-cde9-49e3-bf6c-8e3434d30639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:58.024740 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47969e7e-cde9-49e3-bf6c-8e3434d30639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:58.025166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47969e7e-cde9-49e3-bf6c-8e3434d30639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:58.025460 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47969e7e-cde9-49e3-bf6c-8e3434d30639 admin admin] PATH_INFO: `/v3/users/aa3150d932224bb0a737a3977fff46bc` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:58.030928 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47969e7e-cde9-49e3-bf6c-8e3434d30639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa3150d932224bb0a737a3977fff46bc)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:58.033472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47969e7e-cde9-49e3-bf6c-8e3434d30639 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:58.110571 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-47969e7e-cde9-49e3-bf6c-8e3434d30639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa3150d932224bb0a737a3977fff46bc'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:58.111233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-47969e7e-cde9-49e3-bf6c-8e3434d30639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa3150d932224bb0a737a3977fff46bc'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:58.119421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-47969e7e-cde9-49e3-bf6c-8e3434d30639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa3150d932224bb0a737a3977fff46bc'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:58.131934 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:58.139161 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-47969e7e-cde9-49e3-bf6c-8e3434d30639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa3150d932224bb0a737a3977fff46bc'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:58.140339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-47969e7e-cde9-49e3-bf6c-8e3434d30639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa3150d932224bb0a737a3977fff46bc'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:58.154787 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1396/2762] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:57 2022] DELETE /identity/v3/users/aa3150d932224bb0a737a3977fff46bc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:58.335888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f062c-087c-4b5d-87c5-17a023e16116 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:58.349454 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f062c-087c-4b5d-87c5-17a023e16116 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:58.413166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92257d-e485-4c5a-a6b4-4066b1e55a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:58.414222 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92257d-e485-4c5a-a6b4-4066b1e55a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:58.414585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92257d-e485-4c5a-a6b4-4066b1e55a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:58.415053 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92257d-e485-4c5a-a6b4-4066b1e55a18 admin admin] PATH_INFO: `/v3/projects/e0ac172df04f4957bdd5003bb71ce010` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:58.416272 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92257d-e485-4c5a-a6b4-4066b1e55a18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0ac172df04f4957bdd5003bb71ce010)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:58.419230 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92257d-e485-4c5a-a6b4-4066b1e55a18 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:58.470098 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5b92257d-e485-4c5a-a6b4-4066b1e55a18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0ac172df04f4957bdd5003bb71ce010', 'request_id': 'req-5b92257d-e485-4c5a-a6b4-4066b1e55a18'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:58.476774 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5b92257d-e485-4c5a-a6b4-4066b1e55a18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0ac172df04f4957bdd5003bb71ce010', 'request_id': 'req-5b92257d-e485-4c5a-a6b4-4066b1e55a18'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:58.485354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5b92257d-e485-4c5a-a6b4-4066b1e55a18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0ac172df04f4957bdd5003bb71ce010', 'request_id': 'req-5b92257d-e485-4c5a-a6b4-4066b1e55a18'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:58.497937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5b92257d-e485-4c5a-a6b4-4066b1e55a18 admin admin] The token cache is being invalidate because project e0ac172df04f4957bdd5003bb71ce010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:12:58.498232 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5b92257d-e485-4c5a-a6b4-4066b1e55a18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:58.499064 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5b92257d-e485-4c5a-a6b4-4066b1e55a18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:58.500609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1367/2763] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:12:58 2022] DELETE /identity/v3/projects/e0ac172df04f4957bdd5003bb71ce010 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:58.506427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ba450-a60f-4cac-af61-35170b32f12c None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:58.506723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ba450-a60f-4cac-af61-35170b32f12c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:58.507065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ba450-a60f-4cac-af61-35170b32f12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:58.525251 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-678ba450-a60f-4cac-af61-35170b32f12c None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:58.527458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-678ba450-a60f-4cac-af61-35170b32f12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:58.593258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1397/2764] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:58.602147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2e097-33b8-4f09-b0f2-21c3cc97d600 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:58.602551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2e097-33b8-4f09-b0f2-21c3cc97d600 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:58.602900 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2e097-33b8-4f09-b0f2-21c3cc97d600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:58.624175 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-92d2e097-33b8-4f09-b0f2-21c3cc97d600 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:58.625423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-92d2e097-33b8-4f09-b0f2-21c3cc97d600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:58.633327 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1368/2765] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:58.638238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ba450-a60f-4cac-af61-35170b32f12c None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:58.648146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ba450-a60f-4cac-af61-35170b32f12c None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:58.653523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19195-3f00-4387-8aca-b9365b946beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:58.654174 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19195-3f00-4387-8aca-b9365b946beb admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:58.654463 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19195-3f00-4387-8aca-b9365b946beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:58.654742 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19195-3f00-4387-8aca-b9365b946beb admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:58.655468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf19195-3f00-4387-8aca-b9365b946beb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:12:58.655776 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf19195-3f00-4387-8aca-b9365b946beb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:58.657394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf19195-3f00-4387-8aca-b9365b946beb admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:58.669302 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1398/2766] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:12:58.673724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2e097-33b8-4f09-b0f2-21c3cc97d600 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:58.683632 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2e097-33b8-4f09-b0f2-21c3cc97d600 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:58.688333 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e602e9-102b-4726-b470-a65c4599d6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:58.689007 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e602e9-102b-4726-b470-a65c4599d6bb admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:58.689367 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e602e9-102b-4726-b470-a65c4599d6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:58.689653 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e602e9-102b-4726-b470-a65c4599d6bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:58.690335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e602e9-102b-4726-b470-a65c4599d6bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:58.692199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e602e9-102b-4726-b470-a65c4599d6bb admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:58.696308 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:58.720216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1369/2767] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:12:58 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:12:58.724820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19195-3f00-4387-8aca-b9365b946beb admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:58.735435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19195-3f00-4387-8aca-b9365b946beb admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:58.795004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602ecb3-b717-439e-8289-468455cd6909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:58.795665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602ecb3-b717-439e-8289-468455cd6909 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:58.795948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602ecb3-b717-439e-8289-468455cd6909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:58.796220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602ecb3-b717-439e-8289-468455cd6909 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:58.796892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602ecb3-b717-439e-8289-468455cd6909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:58.798858 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602ecb3-b717-439e-8289-468455cd6909 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:58.820365 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1399/2768] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:12:58 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:12:58.825252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e602e9-102b-4726-b470-a65c4599d6bb admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:58.835353 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e602e9-102b-4726-b470-a65c4599d6bb admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:58.840042 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4a4bb-6b73-4833-8abe-418f78c5ce6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:58.840694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4a4bb-6b73-4833-8abe-418f78c5ce6c admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:58.840974 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4a4bb-6b73-4833-8abe-418f78c5ce6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:58.841283 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4a4bb-6b73-4833-8abe-418f78c5ce6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:58.841996 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4a4bb-6b73-4833-8abe-418f78c5ce6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:58.843566 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4a4bb-6b73-4833-8abe-418f78c5ce6c admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:58.853981 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1370/2769] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:12:58.857839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602ecb3-b717-439e-8289-468455cd6909 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:58.867836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602ecb3-b717-439e-8289-468455cd6909 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:58.875065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5aaa4b-3db1-46fa-af06-c6913b8cb66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:58.875872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5aaa4b-3db1-46fa-af06-c6913b8cb66c admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:58.876162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5aaa4b-3db1-46fa-af06-c6913b8cb66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:58.876711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5aaa4b-3db1-46fa-af06-c6913b8cb66c admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:58.877500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5aaa4b-3db1-46fa-af06-c6913b8cb66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:58.879110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5aaa4b-3db1-46fa-af06-c6913b8cb66c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:58.890468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1400/2770] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:12:58.895495 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4a4bb-6b73-4833-8abe-418f78c5ce6c admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:58.905503 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4a4bb-6b73-4833-8abe-418f78c5ce6c admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:58.910147 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f6ae8-b81e-44ad-8fd0-7313c2101f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:58.911024 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f6ae8-b81e-44ad-8fd0-7313c2101f6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:58.911306 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f6ae8-b81e-44ad-8fd0-7313c2101f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:58.911584 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f6ae8-b81e-44ad-8fd0-7313c2101f6b admin admin] PATH_INFO: `/v3/projects/0c6df372585b482c8711971205109b8b/users/e47343a169b040569689892d6aa2cac2/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:58.926399 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f6ae8-b81e-44ad-8fd0-7313c2101f6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c6df372585b482c8711971205109b8b, user_id=e47343a169b040569689892d6aa2cac2, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:58.929027 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f6ae8-b81e-44ad-8fd0-7313c2101f6b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:58.942337 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1371/2771] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:12:58 2022] PUT /identity/v3/projects/0c6df372585b482c8711971205109b8b/users/e47343a169b040569689892d6aa2cac2/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:58.953406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5de4e-ad53-46e1-b881-3931f3536f5e None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:58.953699 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5de4e-ad53-46e1-b881-3931f3536f5e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:58.953978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5de4e-ad53-46e1-b881-3931f3536f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:58.980134 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-d0c5de4e-ad53-46e1-b881-3931f3536f5e None None] MFA Rules not processed for user `e47343a169b040569689892d6aa2cac2`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:58.981493 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d0c5de4e-ad53-46e1-b881-3931f3536f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:59.023766 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:12:59.052025 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1401/2772] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:12:59.063817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f6ae8-b81e-44ad-8fd0-7313c2101f6b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:59.065623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3b3f6ae8-b81e-44ad-8fd0-7313c2101f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:59.145370 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f6ae8-b81e-44ad-8fd0-7313c2101f6b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:59.151110 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d6598-a72e-40f0-b0d7-f46e11941309 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:59.151837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d6598-a72e-40f0-b0d7-f46e11941309 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:59.152122 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d6598-a72e-40f0-b0d7-f46e11941309 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:59.152399 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d6598-a72e-40f0-b0d7-f46e11941309 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:59.160405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d6598-a72e-40f0-b0d7-f46e11941309 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:59.162079 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d6598-a72e-40f0-b0d7-f46e11941309 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:59.171841 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1372/2773] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:12:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:12:59.774034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5de4e-ad53-46e1-b881-3931f3536f5e None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:12:59.775467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d0c5de4e-ad53-46e1-b881-3931f3536f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:12:59.841294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5de4e-ad53-46e1-b881-3931f3536f5e None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:12:59.846077 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:12:59.846941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:59.847226 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:59.847502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] PATH_INFO: `/v3/users/c1331c3bfca64ad68a1a500150e9f4d0` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:59.848680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1331c3bfca64ad68a1a500150e9f4d0)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:12:59.850661 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:12:59.897229 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1331c3bfca64ad68a1a500150e9f4d0'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:59.897874 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1331c3bfca64ad68a1a500150e9f4d0'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:59.904822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1331c3bfca64ad68a1a500150e9f4d0'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:59.916837 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1331c3bfca64ad68a1a500150e9f4d0'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:59.917503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1331c3bfca64ad68a1a500150e9f4d0'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:12:59.929739 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1402/2774] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:12:59 2022] DELETE /identity/v3/users/c1331c3bfca64ad68a1a500150e9f4d0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:12:59.934948 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45b2cb-e8cc-4c07-95b3-f1a695e6fb4f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:12:59.935366 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45b2cb-e8cc-4c07-95b3-f1a695e6fb4f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:12:59.935624 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45b2cb-e8cc-4c07-95b3-f1a695e6fb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:12:59.952544 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-7a45b2cb-e8cc-4c07-95b3-f1a695e6fb4f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:12:59.953814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7a45b2cb-e8cc-4c07-95b3-f1a695e6fb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:00.015554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1373/2775] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:00.025074 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:00.026751 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:00.084241 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145f67c-e79a-44ad-9448-376c5aba8811 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:00.087906 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a8113-7a36-4e2a-a816-0912efe490d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:00.088566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a8113-7a36-4e2a-a816-0912efe490d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:00.088849 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a8113-7a36-4e2a-a816-0912efe490d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:00.089159 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a8113-7a36-4e2a-a816-0912efe490d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:00.098412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a8113-7a36-4e2a-a816-0912efe490d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:00.099839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a8113-7a36-4e2a-a816-0912efe490d7 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:00.109829 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1403/2776] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:13:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:13:00.375437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45b2cb-e8cc-4c07-95b3-f1a695e6fb4f None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:00.377018 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7a45b2cb-e8cc-4c07-95b3-f1a695e6fb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:00.389542 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45b2cb-e8cc-4c07-95b3-f1a695e6fb4f None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:00.394354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:00.395128 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:00.395407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:00.395683 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] PATH_INFO: `/v3/projects/c1ba3e44a1dc4b4b828a173e9471bec9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:00.396874 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ba3e44a1dc4b4b828a173e9471bec9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:00.398828 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:00.435498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:00.448529 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ba3e44a1dc4b4b828a173e9471bec9', 'request_id': 'req-807ddc01-e3ea-4231-877f-2d60873a0979'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:00.454164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ba3e44a1dc4b4b828a173e9471bec9', 'request_id': 'req-807ddc01-e3ea-4231-877f-2d60873a0979'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:00.461446 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ba3e44a1dc4b4b828a173e9471bec9', 'request_id': 'req-807ddc01-e3ea-4231-877f-2d60873a0979'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:00.473428 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] The token cache is being invalidate because project c1ba3e44a1dc4b4b828a173e9471bec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:13:00.473718 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:00.474340 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:00.475870 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1374/2777] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:13:00 2022] DELETE /identity/v3/projects/c1ba3e44a1dc4b4b828a173e9471bec9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:00.480079 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a8113-7a36-4e2a-a816-0912efe490d7 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:00.484887 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:00.491479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a8113-7a36-4e2a-a816-0912efe490d7 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:00.548494 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490f973-5426-4da5-ae53-2a8b65e1a5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:00.549429 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490f973-5426-4da5-ae53-2a8b65e1a5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:00.549705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490f973-5426-4da5-ae53-2a8b65e1a5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:00.550010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490f973-5426-4da5-ae53-2a8b65e1a5bb admin admin] PATH_INFO: `/v3/users/bbbefd227437470dbdd5b2861b2b152c` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:00.551218 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490f973-5426-4da5-ae53-2a8b65e1a5bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbbefd227437470dbdd5b2861b2b152c)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:00.553147 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490f973-5426-4da5-ae53-2a8b65e1a5bb admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:00.605476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b490f973-5426-4da5-ae53-2a8b65e1a5bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbbefd227437470dbdd5b2861b2b152c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:00.606146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b490f973-5426-4da5-ae53-2a8b65e1a5bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbbefd227437470dbdd5b2861b2b152c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:00.613309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b490f973-5426-4da5-ae53-2a8b65e1a5bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbbefd227437470dbdd5b2861b2b152c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:00.626107 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b490f973-5426-4da5-ae53-2a8b65e1a5bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbbefd227437470dbdd5b2861b2b152c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:00.626800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-b490f973-5426-4da5-ae53-2a8b65e1a5bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbbefd227437470dbdd5b2861b2b152c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:00.641029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1404/2778] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:13:00 2022] DELETE /identity/v3/users/bbbefd227437470dbdd5b2861b2b152c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:13:00.938417 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:00.939864 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:00.953860 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ddc01-e3ea-4231-877f-2d60873a0979 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:01.014990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:01.015849 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:01.016129 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:01.016402 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0 admin admin] PATH_INFO: `/v3/projects/017e18d18da94630b262454378906e7a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:01.017737 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=017e18d18da94630b262454378906e7a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:01.019667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:01.064023 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '017e18d18da94630b262454378906e7a', 'request_id': 'req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:01.069335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '017e18d18da94630b262454378906e7a', 'request_id': 'req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:01.076737 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '017e18d18da94630b262454378906e7a', 'request_id': 'req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:01.087486 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0 admin admin] The token cache is being invalidate because project 017e18d18da94630b262454378906e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:13:01.087770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:01.088422 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-7d7d2fc4-f05a-4fc1-b0dd-ffddec756ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:01.089976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1375/2779] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:13:00 2022] DELETE /identity/v3/projects/017e18d18da94630b262454378906e7a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:01.095213 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab6acf-d7f8-42ed-8a39-39a6d5b4198a None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:01.095957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab6acf-d7f8-42ed-8a39-39a6d5b4198a None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:01.096069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab6acf-d7f8-42ed-8a39-39a6d5b4198a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:01.113324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-02ab6acf-d7f8-42ed-8a39-39a6d5b4198a None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:01.114523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-02ab6acf-d7f8-42ed-8a39-39a6d5b4198a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:01.173052 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1405/2780] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:01.180747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d107b3b-d212-4a48-80b6-5a2c79dad3e4 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:01.181043 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d107b3b-d212-4a48-80b6-5a2c79dad3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:01.181384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d107b3b-d212-4a48-80b6-5a2c79dad3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:01.200458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9d107b3b-d212-4a48-80b6-5a2c79dad3e4 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:01.201683 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9d107b3b-d212-4a48-80b6-5a2c79dad3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:01.209582 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1376/2781] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:01.213659 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab6acf-d7f8-42ed-8a39-39a6d5b4198a None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:01.224482 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab6acf-d7f8-42ed-8a39-39a6d5b4198a None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:01.231619 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4f4ca-3732-47b5-a28e-b0e5fff7abda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:01.232273 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4f4ca-3732-47b5-a28e-b0e5fff7abda admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:01.233195 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4f4ca-3732-47b5-a28e-b0e5fff7abda admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:01.233479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4f4ca-3732-47b5-a28e-b0e5fff7abda admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:01.234271 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4f4ca-3732-47b5-a28e-b0e5fff7abda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:13:01.234577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4f4ca-3732-47b5-a28e-b0e5fff7abda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:01.236772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4f4ca-3732-47b5-a28e-b0e5fff7abda admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:01.250003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1406/2782] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:13:01.254242 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d107b3b-d212-4a48-80b6-5a2c79dad3e4 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:01.264317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d107b3b-d212-4a48-80b6-5a2c79dad3e4 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:01.268980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28cac7-7900-4ae2-9916-cc5c7d53ef53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:01.269666 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28cac7-7900-4ae2-9916-cc5c7d53ef53 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:01.269978 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28cac7-7900-4ae2-9916-cc5c7d53ef53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:01.270257 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28cac7-7900-4ae2-9916-cc5c7d53ef53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:01.270938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28cac7-7900-4ae2-9916-cc5c7d53ef53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:01.272814 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28cac7-7900-4ae2-9916-cc5c7d53ef53 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:01.298034 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1377/2783] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:13:01 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:13:01.302014 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4f4ca-3732-47b5-a28e-b0e5fff7abda admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:01.312421 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4f4ca-3732-47b5-a28e-b0e5fff7abda admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:01.317054 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:01.372175 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104c142-49ab-4c1e-9124-a2fda14b4e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:01.373446 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104c142-49ab-4c1e-9124-a2fda14b4e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:01.373980 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104c142-49ab-4c1e-9124-a2fda14b4e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:01.374483 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104c142-49ab-4c1e-9124-a2fda14b4e93 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:01.375326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104c142-49ab-4c1e-9124-a2fda14b4e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:01.377877 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104c142-49ab-4c1e-9124-a2fda14b4e93 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:01.404213 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1407/2784] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:13:01 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:13:01.408093 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28cac7-7900-4ae2-9916-cc5c7d53ef53 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:01.418404 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28cac7-7900-4ae2-9916-cc5c7d53ef53 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:01.423027 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f06033-97e1-4780-be6e-210f4606def7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:01.423672 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f06033-97e1-4780-be6e-210f4606def7 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:01.423951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f06033-97e1-4780-be6e-210f4606def7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:01.424236 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f06033-97e1-4780-be6e-210f4606def7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:01.424936 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f06033-97e1-4780-be6e-210f4606def7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:01.426545 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f06033-97e1-4780-be6e-210f4606def7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:01.437305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1378/2785] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:13:01.441240 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104c142-49ab-4c1e-9124-a2fda14b4e93 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:01.451828 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104c142-49ab-4c1e-9124-a2fda14b4e93 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:01.456500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef1af5-3f94-43ad-98b2-422a60187468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:01.457219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef1af5-3f94-43ad-98b2-422a60187468 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:01.457510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef1af5-3f94-43ad-98b2-422a60187468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:01.457790 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef1af5-3f94-43ad-98b2-422a60187468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:01.458513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef1af5-3f94-43ad-98b2-422a60187468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:01.460118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef1af5-3f94-43ad-98b2-422a60187468 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:01.471047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1408/2786] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:13:01.474989 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f06033-97e1-4780-be6e-210f4606def7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:01.485107 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f06033-97e1-4780-be6e-210f4606def7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:01.489735 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d2e98-d787-422f-8e2e-dae64e530ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:01.490603 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d2e98-d787-422f-8e2e-dae64e530ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:01.490887 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d2e98-d787-422f-8e2e-dae64e530ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:01.491168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d2e98-d787-422f-8e2e-dae64e530ac3 admin admin] PATH_INFO: `/v3/projects/0bcc86c7c70944c483c364c853fd3e25/users/b5940aa7c6f64e2c85702f69337b432b/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:01.499282 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:01.507506 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d2e98-d787-422f-8e2e-dae64e530ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bcc86c7c70944c483c364c853fd3e25, user_id=b5940aa7c6f64e2c85702f69337b432b, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:01.511104 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d2e98-d787-422f-8e2e-dae64e530ac3 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:01.524052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1379/2787] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:13:01 2022] PUT /identity/v3/projects/0bcc86c7c70944c483c364c853fd3e25/users/b5940aa7c6f64e2c85702f69337b432b/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:01.534904 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d001a6c-fad5-4a73-89ac-89f2eb241059 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:01.535198 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d001a6c-fad5-4a73-89ac-89f2eb241059 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:01.535476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d001a6c-fad5-4a73-89ac-89f2eb241059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:01.562556 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-0d001a6c-fad5-4a73-89ac-89f2eb241059 None None] MFA Rules not processed for user `b5940aa7c6f64e2c85702f69337b432b`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:01.563760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0d001a6c-fad5-4a73-89ac-89f2eb241059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:01.625848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1409/2788] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:01.637462 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d2e98-d787-422f-8e2e-dae64e530ac3 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:01.639237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b67d2e98-d787-422f-8e2e-dae64e530ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:01.696909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d2e98-d787-422f-8e2e-dae64e530ac3 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:01.700626 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c4660-ad7b-4d92-97bd-bd97aa077b99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:01.701301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c4660-ad7b-4d92-97bd-bd97aa077b99 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:01.701585 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c4660-ad7b-4d92-97bd-bd97aa077b99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:01.701861 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c4660-ad7b-4d92-97bd-bd97aa077b99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:01.710316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c4660-ad7b-4d92-97bd-bd97aa077b99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:01.711747 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c4660-ad7b-4d92-97bd-bd97aa077b99 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:01.722083 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1380/2789] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:13:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:13:01.742872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d001a6c-fad5-4a73-89ac-89f2eb241059 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:01.803873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d001a6c-fad5-4a73-89ac-89f2eb241059 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:01.808873 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:01.809772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:01.810066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:01.810349 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] PATH_INFO: `/v3/users/b5940aa7c6f64e2c85702f69337b432b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:01.811553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5940aa7c6f64e2c85702f69337b432b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:01.813541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:01.834201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:01.861461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5940aa7c6f64e2c85702f69337b432b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:01.862123 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5940aa7c6f64e2c85702f69337b432b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:01.868861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5940aa7c6f64e2c85702f69337b432b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:01.880350 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5940aa7c6f64e2c85702f69337b432b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:01.881001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5940aa7c6f64e2c85702f69337b432b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:01.894593 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1410/2790] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:13:01 2022] DELETE /identity/v3/users/b5940aa7c6f64e2c85702f69337b432b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:01.899390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761537ac-a0c1-4706-880d-db9a79644148 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:01.899390 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761537ac-a0c1-4706-880d-db9a79644148 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:01.899668 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761537ac-a0c1-4706-880d-db9a79644148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:01.918787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-761537ac-a0c1-4706-880d-db9a79644148 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:01.919842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-761537ac-a0c1-4706-880d-db9a79644148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:01.990458 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1381/2791] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:02.000003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:02.001458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:02.058838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421814b-abd2-4cd5-9892-a5a06254eda5 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:02.062549 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6b3f5-62d3-4c83-ad69-c22ea9e5f20e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:02.063190 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6b3f5-62d3-4c83-ad69-c22ea9e5f20e service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:02.063473 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6b3f5-62d3-4c83-ad69-c22ea9e5f20e service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:02.063748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6b3f5-62d3-4c83-ad69-c22ea9e5f20e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:02.073237 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6b3f5-62d3-4c83-ad69-c22ea9e5f20e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:02.074771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6b3f5-62d3-4c83-ad69-c22ea9e5f20e service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:02.084124 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1411/2792] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:13:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:13:02.351705 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761537ac-a0c1-4706-880d-db9a79644148 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:02.353273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-761537ac-a0c1-4706-880d-db9a79644148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:02.366313 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761537ac-a0c1-4706-880d-db9a79644148 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:02.371216 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824dce55-506c-42a1-a435-bea1d07edfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:02.372084 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dce55-506c-42a1-a435-bea1d07edfbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:02.372369 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dce55-506c-42a1-a435-bea1d07edfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:02.372648 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dce55-506c-42a1-a435-bea1d07edfbc admin admin] PATH_INFO: `/v3/projects/0bcc86c7c70944c483c364c853fd3e25` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:02.373862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824dce55-506c-42a1-a435-bea1d07edfbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bcc86c7c70944c483c364c853fd3e25)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:02.375773 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824dce55-506c-42a1-a435-bea1d07edfbc admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:02.423702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-824dce55-506c-42a1-a435-bea1d07edfbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bcc86c7c70944c483c364c853fd3e25', 'request_id': 'req-824dce55-506c-42a1-a435-bea1d07edfbc'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:02.430165 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-824dce55-506c-42a1-a435-bea1d07edfbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bcc86c7c70944c483c364c853fd3e25', 'request_id': 'req-824dce55-506c-42a1-a435-bea1d07edfbc'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:02.438242 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-824dce55-506c-42a1-a435-bea1d07edfbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bcc86c7c70944c483c364c853fd3e25', 'request_id': 'req-824dce55-506c-42a1-a435-bea1d07edfbc'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:02.450077 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-824dce55-506c-42a1-a435-bea1d07edfbc admin admin] The token cache is being invalidate because project 0bcc86c7c70944c483c364c853fd3e25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:13:02.450361 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-824dce55-506c-42a1-a435-bea1d07edfbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:02.450993 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-824dce55-506c-42a1-a435-bea1d07edfbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:02.452571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1382/2793] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:13:02 2022] DELETE /identity/v3/projects/0bcc86c7c70944c483c364c853fd3e25 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:02.458516 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149caa44-5f56-4880-8d8f-7f77abb7d4b6 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:02.458971 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149caa44-5f56-4880-8d8f-7f77abb7d4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:02.459083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149caa44-5f56-4880-8d8f-7f77abb7d4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:02.476383 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-149caa44-5f56-4880-8d8f-7f77abb7d4b6 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:02.477602 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-149caa44-5f56-4880-8d8f-7f77abb7d4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:02.541465 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1412/2794] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:02.549328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5f03a-d7f2-4942-9f78-3e093edac091 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:02.549670 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5f03a-d7f2-4942-9f78-3e093edac091 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:02.549945 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5f03a-d7f2-4942-9f78-3e093edac091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:02.569915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0db5f03a-d7f2-4942-9f78-3e093edac091 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:02.571112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0db5f03a-d7f2-4942-9f78-3e093edac091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:02.579605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1383/2795] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:13:02.583763 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149caa44-5f56-4880-8d8f-7f77abb7d4b6 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:02.593475 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149caa44-5f56-4880-8d8f-7f77abb7d4b6 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:02.598241 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b4cc1-8da1-4bd5-9a1e-f01c4f9a8392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:02.598887 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b4cc1-8da1-4bd5-9a1e-f01c4f9a8392 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:02.599196 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b4cc1-8da1-4bd5-9a1e-f01c4f9a8392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:02.599472 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b4cc1-8da1-4bd5-9a1e-f01c4f9a8392 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:02.600219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b4cc1-8da1-4bd5-9a1e-f01c4f9a8392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:13:02.600527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b4cc1-8da1-4bd5-9a1e-f01c4f9a8392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:02.602151 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b4cc1-8da1-4bd5-9a1e-f01c4f9a8392 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:02.613614 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1413/2796] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:13:02.617694 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5f03a-d7f2-4942-9f78-3e093edac091 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:02.627903 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5f03a-d7f2-4942-9f78-3e093edac091 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:02.632512 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415568d-0857-40e6-ab16-b07918b5683b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:02.633189 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415568d-0857-40e6-ab16-b07918b5683b admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:02.633475 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415568d-0857-40e6-ab16-b07918b5683b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:02.633753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415568d-0857-40e6-ab16-b07918b5683b admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:02.634441 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415568d-0857-40e6-ab16-b07918b5683b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:02.636934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415568d-0857-40e6-ab16-b07918b5683b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:02.640183 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:02.663684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1384/2797] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:13:02 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:13:02.667792 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b4cc1-8da1-4bd5-9a1e-f01c4f9a8392 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:02.677610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b4cc1-8da1-4bd5-9a1e-f01c4f9a8392 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:02.702999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:02.736780 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c4a35-b8bd-4957-9b64-d20d4fc11940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:02.737476 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c4a35-b8bd-4957-9b64-d20d4fc11940 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:02.737759 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c4a35-b8bd-4957-9b64-d20d4fc11940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:02.738034 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c4a35-b8bd-4957-9b64-d20d4fc11940 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:02.739206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c4a35-b8bd-4957-9b64-d20d4fc11940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:02.740618 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c4a35-b8bd-4957-9b64-d20d4fc11940 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:02.761427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1414/2798] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:13:02 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:13:02.765493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415568d-0857-40e6-ab16-b07918b5683b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:02.775731 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415568d-0857-40e6-ab16-b07918b5683b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:02.780352 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa920c0-7602-486c-b5c2-712a26c293e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:02.781006 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa920c0-7602-486c-b5c2-712a26c293e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:02.781328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa920c0-7602-486c-b5c2-712a26c293e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:02.781609 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa920c0-7602-486c-b5c2-712a26c293e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:02.782328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa920c0-7602-486c-b5c2-712a26c293e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:02.783906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa920c0-7602-486c-b5c2-712a26c293e9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:02.794249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1385/2799] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:13:02.798052 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c4a35-b8bd-4957-9b64-d20d4fc11940 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:02.809447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c4a35-b8bd-4957-9b64-d20d4fc11940 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:02.814053 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1d0a8-953c-435f-9851-99bf6f5442da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:02.814715 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1d0a8-953c-435f-9851-99bf6f5442da admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:02.815003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1d0a8-953c-435f-9851-99bf6f5442da admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:02.815283 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1d0a8-953c-435f-9851-99bf6f5442da admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:02.816011 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1d0a8-953c-435f-9851-99bf6f5442da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:02.817671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1d0a8-953c-435f-9851-99bf6f5442da admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:02.829801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1415/2800] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:13:02.834049 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa920c0-7602-486c-b5c2-712a26c293e9 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:02.844125 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa920c0-7602-486c-b5c2-712a26c293e9 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:02.849181 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c994b44-02f7-4af2-80b5-59e94649d848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:02.850067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c994b44-02f7-4af2-80b5-59e94649d848 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:02.850351 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c994b44-02f7-4af2-80b5-59e94649d848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:02.850629 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c994b44-02f7-4af2-80b5-59e94649d848 admin admin] PATH_INFO: `/v3/projects/4a67265e323b4de19c2db01b39652f20/users/35c433f2b7b34ddebb147b103ad29a3d/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:02.865550 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c994b44-02f7-4af2-80b5-59e94649d848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a67265e323b4de19c2db01b39652f20, user_id=35c433f2b7b34ddebb147b103ad29a3d, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:02.868201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c994b44-02f7-4af2-80b5-59e94649d848 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:02.881712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1386/2801] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:13:02 2022] PUT /identity/v3/projects/4a67265e323b4de19c2db01b39652f20/users/35c433f2b7b34ddebb147b103ad29a3d/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:02.892214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd1342-a4c3-4291-a37b-ec893b97d3b3 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:02.892510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd1342-a4c3-4291-a37b-ec893b97d3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:02.892793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd1342-a4c3-4291-a37b-ec893b97d3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:02.919539 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-4ecd1342-a4c3-4291-a37b-ec893b97d3b3 None None] MFA Rules not processed for user `35c433f2b7b34ddebb147b103ad29a3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:02.920772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4ecd1342-a4c3-4291-a37b-ec893b97d3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:02.986728 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1416/2802] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:02.999075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c994b44-02f7-4af2-80b5-59e94649d848 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:03.000805 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0c994b44-02f7-4af2-80b5-59e94649d848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:03.057607 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c994b44-02f7-4af2-80b5-59e94649d848 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:03.061221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938113a-7ad4-44ef-8db2-8cc418785ceb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:03.061866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938113a-7ad4-44ef-8db2-8cc418785ceb service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:03.062152 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938113a-7ad4-44ef-8db2-8cc418785ceb service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:03.062432 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938113a-7ad4-44ef-8db2-8cc418785ceb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:03.070845 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938113a-7ad4-44ef-8db2-8cc418785ceb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:03.072271 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938113a-7ad4-44ef-8db2-8cc418785ceb service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:03.081046 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1387/2803] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:13:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:13:03.106503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd1342-a4c3-4291-a37b-ec893b97d3b3 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:03.168861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd1342-a4c3-4291-a37b-ec893b97d3b3 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:03.173502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:03.174376 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:03.174663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:03.174945 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] PATH_INFO: `/v3/users/35c433f2b7b34ddebb147b103ad29a3d` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:03.176141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c433f2b7b34ddebb147b103ad29a3d)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:03.178093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:03.216658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:03.227194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c433f2b7b34ddebb147b103ad29a3d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:03.227875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c433f2b7b34ddebb147b103ad29a3d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:03.235510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c433f2b7b34ddebb147b103ad29a3d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:03.247588 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c433f2b7b34ddebb147b103ad29a3d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:03.248257 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c433f2b7b34ddebb147b103ad29a3d'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:03.261675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1417/2804] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:13:03 2022] DELETE /identity/v3/users/35c433f2b7b34ddebb147b103ad29a3d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:03.266138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ef4b6-2f16-4705-a3b2-e13d4a6465a3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:03.266428 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ef4b6-2f16-4705-a3b2-e13d4a6465a3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:03.266704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ef4b6-2f16-4705-a3b2-e13d4a6465a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:03.283952 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-686ef4b6-2f16-4705-a3b2-e13d4a6465a3 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:03.285185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-686ef4b6-2f16-4705-a3b2-e13d4a6465a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:03.346926 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1388/2805] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:03.357046 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:03.358461 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:03.413599 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b128785-ba4b-4c77-8418-9d0b1723951c admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:03.417925 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c9b5b-ffc9-44c7-9497-8ac7462715b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:03.418555 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c9b5b-ffc9-44c7-9497-8ac7462715b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:03.418836 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c9b5b-ffc9-44c7-9497-8ac7462715b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:03.419111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c9b5b-ffc9-44c7-9497-8ac7462715b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:03.428043 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c9b5b-ffc9-44c7-9497-8ac7462715b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:03.429492 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c9b5b-ffc9-44c7-9497-8ac7462715b8 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:03.438726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1418/2806] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:13:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:13:03.706297 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ef4b6-2f16-4705-a3b2-e13d4a6465a3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:03.707837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-686ef4b6-2f16-4705-a3b2-e13d4a6465a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:03.729415 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ef4b6-2f16-4705-a3b2-e13d4a6465a3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:03.734316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e17fff-1c23-45f6-bd02-7210b80aa392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:03.735217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e17fff-1c23-45f6-bd02-7210b80aa392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:03.735518 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e17fff-1c23-45f6-bd02-7210b80aa392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:03.735796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e17fff-1c23-45f6-bd02-7210b80aa392 admin admin] PATH_INFO: `/v3/projects/4a67265e323b4de19c2db01b39652f20` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:03.736973 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e17fff-1c23-45f6-bd02-7210b80aa392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a67265e323b4de19c2db01b39652f20)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:03.738888 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e17fff-1c23-45f6-bd02-7210b80aa392 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:03.783778 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:03.785029 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b1e17fff-1c23-45f6-bd02-7210b80aa392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a67265e323b4de19c2db01b39652f20', 'request_id': 'req-b1e17fff-1c23-45f6-bd02-7210b80aa392'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:03.791977 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b1e17fff-1c23-45f6-bd02-7210b80aa392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a67265e323b4de19c2db01b39652f20', 'request_id': 'req-b1e17fff-1c23-45f6-bd02-7210b80aa392'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:03.799158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b1e17fff-1c23-45f6-bd02-7210b80aa392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a67265e323b4de19c2db01b39652f20', 'request_id': 'req-b1e17fff-1c23-45f6-bd02-7210b80aa392'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:03.810516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b1e17fff-1c23-45f6-bd02-7210b80aa392 admin admin] The token cache is being invalidate because project 4a67265e323b4de19c2db01b39652f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:13:03.810797 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b1e17fff-1c23-45f6-bd02-7210b80aa392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:03.816101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-b1e17fff-1c23-45f6-bd02-7210b80aa392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:03.820767 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1389/2807] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:13:03 2022] DELETE /identity/v3/projects/4a67265e323b4de19c2db01b39652f20 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:13:06.460065 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c9b5b-ffc9-44c7-9497-8ac7462715b8 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:06.460856 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-bf4c9b5b-ffc9-44c7-9497-8ac7462715b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:06.474275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c9b5b-ffc9-44c7-9497-8ac7462715b8 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:06.530100 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:06.530880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:06.531160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:06.531431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] PATH_INFO: `/v3/users/e47343a169b040569689892d6aa2cac2` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:06.532604 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e47343a169b040569689892d6aa2cac2)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:06.534539 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:06.580799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e47343a169b040569689892d6aa2cac2'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:06.581471 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e47343a169b040569689892d6aa2cac2'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:06.588918 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e47343a169b040569689892d6aa2cac2'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:06.590771 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:06.602492 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e47343a169b040569689892d6aa2cac2'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:06.603126 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e47343a169b040569689892d6aa2cac2'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:06.615368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1419/2808] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:13:06 2022] DELETE /identity/v3/users/e47343a169b040569689892d6aa2cac2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:06.620008 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e07a3-4e91-4f0a-b405-73d84b27b099 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:06.620498 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e07a3-4e91-4f0a-b405-73d84b27b099 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:06.620602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e07a3-4e91-4f0a-b405-73d84b27b099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:06.639040 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4c2e07a3-4e91-4f0a-b405-73d84b27b099 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:06.640239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4c2e07a3-4e91-4f0a-b405-73d84b27b099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:06.696497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1390/2809] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:06.706519 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:06.707899 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:06.764948 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6d467-4e9d-4c6f-b78a-e57d881ce537 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:06.768715 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258d3e6-98d8-4cf1-a196-9d95b4bda5b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:06.769403 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258d3e6-98d8-4cf1-a196-9d95b4bda5b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:06.769692 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258d3e6-98d8-4cf1-a196-9d95b4bda5b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:06.769971 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258d3e6-98d8-4cf1-a196-9d95b4bda5b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:06.780010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258d3e6-98d8-4cf1-a196-9d95b4bda5b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:06.781564 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258d3e6-98d8-4cf1-a196-9d95b4bda5b0 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:06.791009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1420/2810] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:13:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:13:07.058377 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e07a3-4e91-4f0a-b405-73d84b27b099 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:07.059907 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4c2e07a3-4e91-4f0a-b405-73d84b27b099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:07.072388 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e07a3-4e91-4f0a-b405-73d84b27b099 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:07.077818 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced6d86-c0ce-4c21-88ae-34149157753d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:07.078796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced6d86-c0ce-4c21-88ae-34149157753d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:07.079144 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced6d86-c0ce-4c21-88ae-34149157753d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:07.079434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced6d86-c0ce-4c21-88ae-34149157753d admin admin] PATH_INFO: `/v3/projects/0c6df372585b482c8711971205109b8b` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:07.080628 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced6d86-c0ce-4c21-88ae-34149157753d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c6df372585b482c8711971205109b8b)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:07.082559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced6d86-c0ce-4c21-88ae-34149157753d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:07.129770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ced6d86-c0ce-4c21-88ae-34149157753d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c6df372585b482c8711971205109b8b', 'request_id': 'req-9ced6d86-c0ce-4c21-88ae-34149157753d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:07.135195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ced6d86-c0ce-4c21-88ae-34149157753d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c6df372585b482c8711971205109b8b', 'request_id': 'req-9ced6d86-c0ce-4c21-88ae-34149157753d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:07.142658 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ced6d86-c0ce-4c21-88ae-34149157753d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c6df372585b482c8711971205109b8b', 'request_id': 'req-9ced6d86-c0ce-4c21-88ae-34149157753d'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:07.154349 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ced6d86-c0ce-4c21-88ae-34149157753d admin admin] The token cache is being invalidate because project 0c6df372585b482c8711971205109b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:13:07.154634 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ced6d86-c0ce-4c21-88ae-34149157753d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:07.155275 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-9ced6d86-c0ce-4c21-88ae-34149157753d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:07.156812 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1391/2811] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:13:07 2022] DELETE /identity/v3/projects/0c6df372585b482c8711971205109b8b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:07.161745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fc564-13b3-4a2d-acb9-cc3d46d2f198 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:07.162040 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fc564-13b3-4a2d-acb9-cc3d46d2f198 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:07.162327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fc564-13b3-4a2d-acb9-cc3d46d2f198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:07.179546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e80fc564-13b3-4a2d-acb9-cc3d46d2f198 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:07.180747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e80fc564-13b3-4a2d-acb9-cc3d46d2f198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:07.239331 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1421/2812] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:07.247480 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156df38-2f83-4c3b-b064-9dd7bd888ff7 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:07.247774 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156df38-2f83-4c3b-b064-9dd7bd888ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:07.248057 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156df38-2f83-4c3b-b064-9dd7bd888ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:07.266779 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-0156df38-2f83-4c3b-b064-9dd7bd888ff7 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:07.267979 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0156df38-2f83-4c3b-b064-9dd7bd888ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:07.275876 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1392/2813] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:13:07.280029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80fc564-13b3-4a2d-acb9-cc3d46d2f198 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:07.289690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80fc564-13b3-4a2d-acb9-cc3d46d2f198 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:07.294781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb32f8c-dcb4-45f8-8a35-8412cb1c54a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:07.295423 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb32f8c-dcb4-45f8-8a35-8412cb1c54a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:07.295710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb32f8c-dcb4-45f8-8a35-8412cb1c54a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:07.296008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb32f8c-dcb4-45f8-8a35-8412cb1c54a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:07.296710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb32f8c-dcb4-45f8-8a35-8412cb1c54a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:13:07.297019 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb32f8c-dcb4-45f8-8a35-8412cb1c54a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:07.298637 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb32f8c-dcb4-45f8-8a35-8412cb1c54a4 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:07.310205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1422/2814] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 09:13:07.314419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156df38-2f83-4c3b-b064-9dd7bd888ff7 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:07.324014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156df38-2f83-4c3b-b064-9dd7bd888ff7 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:07.328637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698daea-2204-4cf4-be31-95ea6e371dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:07.329317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698daea-2204-4cf4-be31-95ea6e371dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:07.329605 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698daea-2204-4cf4-be31-95ea6e371dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:07.329883 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698daea-2204-4cf4-be31-95ea6e371dc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:07.330575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698daea-2204-4cf4-be31-95ea6e371dc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:07.332451 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698daea-2204-4cf4-be31-95ea6e371dc9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:07.358028 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1393/2815] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:13:07 2022] POST /identity/v3/projects => generated 409 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:13:07.362022 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb32f8c-dcb4-45f8-8a35-8412cb1c54a4 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:07.372182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb32f8c-dcb4-45f8-8a35-8412cb1c54a4 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:07.429722 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e9749-0368-48ce-9897-6551a95631e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:07.430399 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e9749-0368-48ce-9897-6551a95631e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:07.430686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e9749-0368-48ce-9897-6551a95631e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:07.430967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e9749-0368-48ce-9897-6551a95631e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:07.431671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e9749-0368-48ce-9897-6551a95631e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:07.433695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e9749-0368-48ce-9897-6551a95631e1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:07.443249 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:07.458544 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1423/2816] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:13:07 2022] POST /identity/v3/users => generated 305 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:13:07.459761 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698daea-2204-4cf4-be31-95ea6e371dc9 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:07.470969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698daea-2204-4cf4-be31-95ea6e371dc9 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:07.475686 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bd24c-b99b-4603-8b76-8df4964f076f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:07.476347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bd24c-b99b-4603-8b76-8df4964f076f admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:07.476635 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bd24c-b99b-4603-8b76-8df4964f076f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:07.476942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bd24c-b99b-4603-8b76-8df4964f076f admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:07.477704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bd24c-b99b-4603-8b76-8df4964f076f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:07.479697 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bd24c-b99b-4603-8b76-8df4964f076f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:07.491042 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1394/2817] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:13:07.495206 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e9749-0368-48ce-9897-6551a95631e1 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:07.505020 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e9749-0368-48ce-9897-6551a95631e1 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:07.511176 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ffc7c-5060-4ac5-b1da-79cc431b367e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:07.511843 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ffc7c-5060-4ac5-b1da-79cc431b367e admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:07.512131 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ffc7c-5060-4ac5-b1da-79cc431b367e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:07.512412 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ffc7c-5060-4ac5-b1da-79cc431b367e admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:07.513173 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ffc7c-5060-4ac5-b1da-79cc431b367e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:07.514796 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ffc7c-5060-4ac5-b1da-79cc431b367e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:07.524749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1424/2818] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:13:07.529476 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bd24c-b99b-4603-8b76-8df4964f076f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:07.539839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bd24c-b99b-4603-8b76-8df4964f076f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:07.544564 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7b077-6aa7-4876-93b2-75738a532aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:07.545501 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7b077-6aa7-4876-93b2-75738a532aa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:07.545869 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7b077-6aa7-4876-93b2-75738a532aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:07.546148 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7b077-6aa7-4876-93b2-75738a532aa1 admin admin] PATH_INFO: `/v3/projects/73e6c75287884b7a80a786677ffef6e9/users/920b3dac0ab9450fbd7028f9f371b29b/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:07.561533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7b077-6aa7-4876-93b2-75738a532aa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e6c75287884b7a80a786677ffef6e9, user_id=920b3dac0ab9450fbd7028f9f371b29b, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:07.564092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7b077-6aa7-4876-93b2-75738a532aa1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:07.567069 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:07.579763 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1395/2819] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:13:07 2022] PUT /identity/v3/projects/73e6c75287884b7a80a786677ffef6e9/users/920b3dac0ab9450fbd7028f9f371b29b/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:07.638285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d098f7-cd8a-4da2-b3d3-fdcaf31bfe03 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:07.638583 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d098f7-cd8a-4da2-b3d3-fdcaf31bfe03 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:07.638882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d098f7-cd8a-4da2-b3d3-fdcaf31bfe03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:07.668252 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-83d098f7-cd8a-4da2-b3d3-fdcaf31bfe03 None None] MFA Rules not processed for user `920b3dac0ab9450fbd7028f9f371b29b`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:07.669513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-83d098f7-cd8a-4da2-b3d3-fdcaf31bfe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:07.734357 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1425/2820] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:07 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:07.738906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7b077-6aa7-4876-93b2-75738a532aa1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:07.798908 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7b077-6aa7-4876-93b2-75738a532aa1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:07.803495 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f489e-8ffa-4763-baab-ff6ae985fbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:07.804140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f489e-8ffa-4763-baab-ff6ae985fbba admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:07.804423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f489e-8ffa-4763-baab-ff6ae985fbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:07.804697 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f489e-8ffa-4763-baab-ff6ae985fbba admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:07.805397 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f489e-8ffa-4763-baab-ff6ae985fbba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:07.807772 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f489e-8ffa-4763-baab-ff6ae985fbba admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:07.839470 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1396/2821] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:13:07 2022] POST /identity/v3/projects => generated 409 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:13:07.847015 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d098f7-cd8a-4da2-b3d3-fdcaf31bfe03 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:07.858601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d098f7-cd8a-4da2-b3d3-fdcaf31bfe03 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:07.914638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c7cc0-7a70-42c8-9f14-3bd5aa72c008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:07.915310 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c7cc0-7a70-42c8-9f14-3bd5aa72c008 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:07.915595 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c7cc0-7a70-42c8-9f14-3bd5aa72c008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:07.915872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c7cc0-7a70-42c8-9f14-3bd5aa72c008 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:07.917336 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c7cc0-7a70-42c8-9f14-3bd5aa72c008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:07.919250 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c7cc0-7a70-42c8-9f14-3bd5aa72c008 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:07.941890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1426/2822] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:13:07 2022] POST /identity/v3/users => generated 311 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:13:07.943159 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f489e-8ffa-4763-baab-ff6ae985fbba admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:07.953783 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f489e-8ffa-4763-baab-ff6ae985fbba admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:07.958415 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69634225-1393-402d-9b14-fa0bd82fb49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:07.959071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69634225-1393-402d-9b14-fa0bd82fb49f admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:07.959357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69634225-1393-402d-9b14-fa0bd82fb49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:07.959637 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69634225-1393-402d-9b14-fa0bd82fb49f admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:07.960357 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69634225-1393-402d-9b14-fa0bd82fb49f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:07.961973 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69634225-1393-402d-9b14-fa0bd82fb49f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:07.973753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1397/2823] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:13:07.977911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c7cc0-7a70-42c8-9f14-3bd5aa72c008 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:07.995520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c7cc0-7a70-42c8-9f14-3bd5aa72c008 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:08.000400 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a7c30-8d6c-46e4-b68e-361f150e74c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:08.001323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a7c30-8d6c-46e4-b68e-361f150e74c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:08.001618 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a7c30-8d6c-46e4-b68e-361f150e74c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:08.001901 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a7c30-8d6c-46e4-b68e-361f150e74c8 admin admin] PATH_INFO: `/v3/projects/c888640c425549ea9afcfee8a7d1020a/users/fd5bb1eef7904548bd4bc36c77cfa532/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:08.016883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a7c30-8d6c-46e4-b68e-361f150e74c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c888640c425549ea9afcfee8a7d1020a, user_id=fd5bb1eef7904548bd4bc36c77cfa532, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:08.019569 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a7c30-8d6c-46e4-b68e-361f150e74c8 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:08.033378 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1427/2824] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:13:07 2022] PUT /identity/v3/projects/c888640c425549ea9afcfee8a7d1020a/users/fd5bb1eef7904548bd4bc36c77cfa532/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:13:08.037442 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69634225-1393-402d-9b14-fa0bd82fb49f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:08.047910 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69634225-1393-402d-9b14-fa0bd82fb49f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:08.103882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90400a-3ce8-4f03-9f19-8e48ed91346d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:08.104468 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90400a-3ce8-4f03-9f19-8e48ed91346d admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:08.104752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90400a-3ce8-4f03-9f19-8e48ed91346d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:08.105026 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90400a-3ce8-4f03-9f19-8e48ed91346d admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:08.105776 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90400a-3ce8-4f03-9f19-8e48ed91346d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:08.107346 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90400a-3ce8-4f03-9f19-8e48ed91346d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:08.117067 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1398/2825] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:13:08.121296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a7c30-8d6c-46e4-b68e-361f150e74c8 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:08.131409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a7c30-8d6c-46e4-b68e-361f150e74c8 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:08.136622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d961091-5e67-4697-9c18-afb48eb32ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:08.137533 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d961091-5e67-4697-9c18-afb48eb32ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:08.137820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d961091-5e67-4697-9c18-afb48eb32ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:08.138099 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d961091-5e67-4697-9c18-afb48eb32ff9 admin admin] PATH_INFO: `/v3/projects/c888640c425549ea9afcfee8a7d1020a/users/fd5bb1eef7904548bd4bc36c77cfa532/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:08.140042 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d961091-5e67-4697-9c18-afb48eb32ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c888640c425549ea9afcfee8a7d1020a, user_id=fd5bb1eef7904548bd4bc36c77cfa532, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:08.142688 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d961091-5e67-4697-9c18-afb48eb32ff9 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:08.153471 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1428/2826] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:13:08 2022] PUT /identity/v3/projects/c888640c425549ea9afcfee8a7d1020a/users/fd5bb1eef7904548bd4bc36c77cfa532/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:08.163516 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0caf69-0aee-4eb5-9e93-d46dc9e110de None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:08.163811 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0caf69-0aee-4eb5-9e93-d46dc9e110de None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:08.164097 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0caf69-0aee-4eb5-9e93-d46dc9e110de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:08.195606 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9e0caf69-0aee-4eb5-9e93-d46dc9e110de None None] MFA Rules not processed for user `fd5bb1eef7904548bd4bc36c77cfa532`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:08.196880 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9e0caf69-0aee-4eb5-9e93-d46dc9e110de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:08.250368 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:08.269269 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1399/2827] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:08 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:13:08.280696 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d961091-5e67-4697-9c18-afb48eb32ff9 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:08.282414 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-9d961091-5e67-4697-9c18-afb48eb32ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:08.313051 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:08.341296 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d961091-5e67-4697-9c18-afb48eb32ff9 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:08.344876 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc46b2a-fa04-479d-86ef-2d2ae30dae50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:08.345549 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc46b2a-fa04-479d-86ef-2d2ae30dae50 service cinder] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:08.346103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc46b2a-fa04-479d-86ef-2d2ae30dae50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:08.346382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc46b2a-fa04-479d-86ef-2d2ae30dae50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:08.355289 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc46b2a-fa04-479d-86ef-2d2ae30dae50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:08.356665 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc46b2a-fa04-479d-86ef-2d2ae30dae50 service cinder] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:08.365972 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1429/2828] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:13:08 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:13:08.516403 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0caf69-0aee-4eb5-9e93-d46dc9e110de None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:08.526493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0caf69-0aee-4eb5-9e93-d46dc9e110de None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:08.530399 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73364779-5416-4bcf-88f3-8326c6888759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:08.531138 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73364779-5416-4bcf-88f3-8326c6888759 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:08.531425 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73364779-5416-4bcf-88f3-8326c6888759 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:08.531702 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73364779-5416-4bcf-88f3-8326c6888759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:08.592331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73364779-5416-4bcf-88f3-8326c6888759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:08.593806 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73364779-5416-4bcf-88f3-8326c6888759 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:08.602859 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1400/2829] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:13:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:13:16.901337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc46b2a-fa04-479d-86ef-2d2ae30dae50 service cinder] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:16.963560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc46b2a-fa04-479d-86ef-2d2ae30dae50 service cinder] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:16.968284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:16.969182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:16.969473 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:16.969752 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] PATH_INFO: `/v3/users/920b3dac0ab9450fbd7028f9f371b29b` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:16.971309 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=920b3dac0ab9450fbd7028f9f371b29b)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:16.973311 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:17.020219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920b3dac0ab9450fbd7028f9f371b29b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.020803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920b3dac0ab9450fbd7028f9f371b29b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.027586 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920b3dac0ab9450fbd7028f9f371b29b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.040347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920b3dac0ab9450fbd7028f9f371b29b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.041004 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920b3dac0ab9450fbd7028f9f371b29b'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.054622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1430/2830] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:13:16 2022] DELETE /identity/v3/users/920b3dac0ab9450fbd7028f9f371b29b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:17.059331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4d75b-ad52-4d52-96f5-7d7b681a79cf None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:17.059750 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4d75b-ad52-4d52-96f5-7d7b681a79cf None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:17.059955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4d75b-ad52-4d52-96f5-7d7b681a79cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:17.077725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-52a4d75b-ad52-4d52-96f5-7d7b681a79cf None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:17.078952 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-52a4d75b-ad52-4d52-96f5-7d7b681a79cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:17.137953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1401/2831] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:17.148337 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:17.149746 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:17.202651 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:17.207362 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a88f16-5506-4fa7-b41e-debf2a249585 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:17.212177 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340ada5-2c97-41df-8982-1f7f0ebecf34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:17.212834 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340ada5-2c97-41df-8982-1f7f0ebecf34 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:17.213194 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340ada5-2c97-41df-8982-1f7f0ebecf34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:17.213686 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340ada5-2c97-41df-8982-1f7f0ebecf34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:17.223111 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340ada5-2c97-41df-8982-1f7f0ebecf34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:17.224538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340ada5-2c97-41df-8982-1f7f0ebecf34 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:17.234001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1431/2832] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:13:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:13:17.495141 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4d75b-ad52-4d52-96f5-7d7b681a79cf None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:17.496577 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-52a4d75b-ad52-4d52-96f5-7d7b681a79cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:17.508457 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4d75b-ad52-4d52-96f5-7d7b681a79cf None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:17.514203 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:17.515146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:17.515435 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:17.515711 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] PATH_INFO: `/v3/projects/73e6c75287884b7a80a786677ffef6e9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:17.516929 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73e6c75287884b7a80a786677ffef6e9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:17.518894 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:17.566230 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73e6c75287884b7a80a786677ffef6e9', 'request_id': 'req-5d973d4e-d11d-4276-8b70-c83e417e7156'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.571556 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73e6c75287884b7a80a786677ffef6e9', 'request_id': 'req-5d973d4e-d11d-4276-8b70-c83e417e7156'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.579330 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73e6c75287884b7a80a786677ffef6e9', 'request_id': 'req-5d973d4e-d11d-4276-8b70-c83e417e7156'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.590132 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] The token cache is being invalidate because project 73e6c75287884b7a80a786677ffef6e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:13:17.590416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.591040 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.593554 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1402/2833] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:13:17 2022] DELETE /identity/v3/projects/73e6c75287884b7a80a786677ffef6e9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:17.597712 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340ada5-2c97-41df-8982-1f7f0ebecf34 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:17.607179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340ada5-2c97-41df-8982-1f7f0ebecf34 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:17.663006 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec452c84-83f3-496c-8ef2-e13198de9e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:17.663872 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec452c84-83f3-496c-8ef2-e13198de9e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:17.664263 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec452c84-83f3-496c-8ef2-e13198de9e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:17.664545 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec452c84-83f3-496c-8ef2-e13198de9e1b admin admin] PATH_INFO: `/v3/users/fd5bb1eef7904548bd4bc36c77cfa532` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:17.665720 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec452c84-83f3-496c-8ef2-e13198de9e1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5bb1eef7904548bd4bc36c77cfa532)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:17.667611 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec452c84-83f3-496c-8ef2-e13198de9e1b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:17.714197 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-ec452c84-83f3-496c-8ef2-e13198de9e1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5bb1eef7904548bd4bc36c77cfa532'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.714847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-ec452c84-83f3-496c-8ef2-e13198de9e1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5bb1eef7904548bd4bc36c77cfa532'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.722447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-ec452c84-83f3-496c-8ef2-e13198de9e1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5bb1eef7904548bd4bc36c77cfa532'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.733957 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-ec452c84-83f3-496c-8ef2-e13198de9e1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5bb1eef7904548bd4bc36c77cfa532'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.734601 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-ec452c84-83f3-496c-8ef2-e13198de9e1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5bb1eef7904548bd4bc36c77cfa532'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:17.746799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1432/2834] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:13:17 2022] DELETE /identity/v3/users/fd5bb1eef7904548bd4bc36c77cfa532 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:18.020898 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:18.022419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:18.037115 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d973d4e-d11d-4276-8b70-c83e417e7156 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:18.048961 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:18.098444 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1480a-77ae-4692-9e82-161cd107c85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:18.099248 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1480a-77ae-4692-9e82-161cd107c85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:18.099537 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1480a-77ae-4692-9e82-161cd107c85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:18.099819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1480a-77ae-4692-9e82-161cd107c85e admin admin] PATH_INFO: `/v3/projects/c888640c425549ea9afcfee8a7d1020a` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:18.101001 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1480a-77ae-4692-9e82-161cd107c85e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c888640c425549ea9afcfee8a7d1020a)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:18.102937 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1480a-77ae-4692-9e82-161cd107c85e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:18.150529 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6fa1480a-77ae-4692-9e82-161cd107c85e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c888640c425549ea9afcfee8a7d1020a', 'request_id': 'req-6fa1480a-77ae-4692-9e82-161cd107c85e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:18.156488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6fa1480a-77ae-4692-9e82-161cd107c85e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c888640c425549ea9afcfee8a7d1020a', 'request_id': 'req-6fa1480a-77ae-4692-9e82-161cd107c85e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:18.163641 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6fa1480a-77ae-4692-9e82-161cd107c85e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c888640c425549ea9afcfee8a7d1020a', 'request_id': 'req-6fa1480a-77ae-4692-9e82-161cd107c85e'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:18.174535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6fa1480a-77ae-4692-9e82-161cd107c85e admin admin] The token cache is being invalidate because project c888640c425549ea9afcfee8a7d1020a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:13:18.174822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6fa1480a-77ae-4692-9e82-161cd107c85e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:18.175469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-6fa1480a-77ae-4692-9e82-161cd107c85e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:18.177004 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1403/2835] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:13:18 2022] DELETE /identity/v3/projects/c888640c425549ea9afcfee8a7d1020a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:18.181890 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943c628-9bb8-458e-af66-248afed43e1e None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:18.182361 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943c628-9bb8-458e-af66-248afed43e1e None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:18.182458 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943c628-9bb8-458e-af66-248afed43e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:18.199986 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-7943c628-9bb8-458e-af66-248afed43e1e None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:18.201746 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-7943c628-9bb8-458e-af66-248afed43e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:18.260294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1433/2836] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:18.269319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9dcca-ffc2-437f-a992-a04036b1f69d None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:18.269618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9dcca-ffc2-437f-a992-a04036b1f69d None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:18.269902 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9dcca-ffc2-437f-a992-a04036b1f69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:18.289557 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-cad9dcca-ffc2-437f-a992-a04036b1f69d None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:18.290759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-cad9dcca-ffc2-437f-a992-a04036b1f69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:18.298701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1404/2837] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:18.302795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943c628-9bb8-458e-af66-248afed43e1e None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:18.312419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943c628-9bb8-458e-af66-248afed43e1e None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:18.316976 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e037562-41d3-416c-bb1e-ac6b43d9d07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:18.317682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e037562-41d3-416c-bb1e-ac6b43d9d07c admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:18.317967 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e037562-41d3-416c-bb1e-ac6b43d9d07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:18.318243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e037562-41d3-416c-bb1e-ac6b43d9d07c admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:18.318992 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e037562-41d3-416c-bb1e-ac6b43d9d07c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:13:18.319304 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e037562-41d3-416c-bb1e-ac6b43d9d07c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:18.320894 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e037562-41d3-416c-bb1e-ac6b43d9d07c admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:18.328089 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:18.334146 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1434/2838] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:13:18.338343 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9dcca-ffc2-437f-a992-a04036b1f69d None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:18.348347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9dcca-ffc2-437f-a992-a04036b1f69d None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:18.353154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45594167-3fda-43c2-8c39-b53eebe892c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:18.353756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594167-3fda-43c2-8c39-b53eebe892c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:18.354043 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594167-3fda-43c2-8c39-b53eebe892c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:18.354322 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594167-3fda-43c2-8c39-b53eebe892c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:18.355005 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45594167-3fda-43c2-8c39-b53eebe892c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:18.356882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45594167-3fda-43c2-8c39-b53eebe892c9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:18.382192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1405/2839] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:13:18 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:13:18.386141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e037562-41d3-416c-bb1e-ac6b43d9d07c admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:18.396761 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e037562-41d3-416c-bb1e-ac6b43d9d07c admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:18.451840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1109c-8f86-4022-904d-97e14c924fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:18.452528 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1109c-8f86-4022-904d-97e14c924fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:18.452815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1109c-8f86-4022-904d-97e14c924fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:18.453110 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1109c-8f86-4022-904d-97e14c924fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:18.453813 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1109c-8f86-4022-904d-97e14c924fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:18.455746 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1109c-8f86-4022-904d-97e14c924fa2 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:18.477308 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1435/2840] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:13:18 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:13:18.481429 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45594167-3fda-43c2-8c39-b53eebe892c9 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:18.491673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45594167-3fda-43c2-8c39-b53eebe892c9 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:18.496187 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e1880-6d50-47c4-ae67-5f16e36959c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:18.496832 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e1880-6d50-47c4-ae67-5f16e36959c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:18.497139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e1880-6d50-47c4-ae67-5f16e36959c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:18.497416 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e1880-6d50-47c4-ae67-5f16e36959c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:18.498664 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e1880-6d50-47c4-ae67-5f16e36959c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:18.500376 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e1880-6d50-47c4-ae67-5f16e36959c6 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:18.510475 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1406/2841] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:13:18.515212 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1109c-8f86-4022-904d-97e14c924fa2 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:18.525267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1109c-8f86-4022-904d-97e14c924fa2 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:18.529941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ce249-4ae8-4500-9e33-98c436b7ed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:18.530599 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ce249-4ae8-4500-9e33-98c436b7ed95 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:18.530888 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ce249-4ae8-4500-9e33-98c436b7ed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:18.531170 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ce249-4ae8-4500-9e33-98c436b7ed95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:18.531896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ce249-4ae8-4500-9e33-98c436b7ed95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:18.533525 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ce249-4ae8-4500-9e33-98c436b7ed95 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:18.543538 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1436/2842] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:13:18.548114 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e1880-6d50-47c4-ae67-5f16e36959c6 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:18.559154 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e1880-6d50-47c4-ae67-5f16e36959c6 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:18.564514 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a326007-f11b-4b89-ac76-a51ee898af11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:18.565423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a326007-f11b-4b89-ac76-a51ee898af11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:18.565708 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a326007-f11b-4b89-ac76-a51ee898af11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:18.565986 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a326007-f11b-4b89-ac76-a51ee898af11 admin admin] PATH_INFO: `/v3/projects/09200991de3a43d68d209d275a3e08a1/users/f087686a737847c4b506862d9d2bfb34/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:18.580273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a326007-f11b-4b89-ac76-a51ee898af11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09200991de3a43d68d209d275a3e08a1, user_id=f087686a737847c4b506862d9d2bfb34, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:18.582951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a326007-f11b-4b89-ac76-a51ee898af11 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:18.596383 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1407/2843] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:13:18 2022] PUT /identity/v3/projects/09200991de3a43d68d209d275a3e08a1/users/f087686a737847c4b506862d9d2bfb34/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:18.606514 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b6fbb-345f-416d-b840-35ae78dc999c None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:18.606749 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b6fbb-345f-416d-b840-35ae78dc999c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:18.607030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b6fbb-345f-416d-b840-35ae78dc999c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:18.634160 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-964b6fbb-345f-416d-b840-35ae78dc999c None None] MFA Rules not processed for user `f087686a737847c4b506862d9d2bfb34`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:18.635374 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-964b6fbb-345f-416d-b840-35ae78dc999c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:18.700411 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1437/2844] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:18.712576 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a326007-f11b-4b89-ac76-a51ee898af11 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:18.714286 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-7a326007-f11b-4b89-ac76-a51ee898af11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:18.756919 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:18.773164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a326007-f11b-4b89-ac76-a51ee898af11 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:18.776796 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a308202-32f7-44f2-8d1a-6146ac97809c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:18.777541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a308202-32f7-44f2-8d1a-6146ac97809c service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:18.777826 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a308202-32f7-44f2-8d1a-6146ac97809c service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:18.778106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a308202-32f7-44f2-8d1a-6146ac97809c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:18.786477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a308202-32f7-44f2-8d1a-6146ac97809c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:18.787886 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a308202-32f7-44f2-8d1a-6146ac97809c service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:18.797063 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1408/2845] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:13:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 09:13:28.697524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:28.792594 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=8 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:37.302540 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b6fbb-345f-416d-b840-35ae78dc999c None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:37.363762 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b6fbb-345f-416d-b840-35ae78dc999c None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:37.368497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:37.369400 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:37.369707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:37.370005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] PATH_INFO: `/v3/users/f087686a737847c4b506862d9d2bfb34` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:37.371201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f087686a737847c4b506862d9d2bfb34)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:37.373624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:37.425754 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f087686a737847c4b506862d9d2bfb34'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:37.426409 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f087686a737847c4b506862d9d2bfb34'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:37.433233 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f087686a737847c4b506862d9d2bfb34'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:37.444861 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f087686a737847c4b506862d9d2bfb34'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:37.445464 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f087686a737847c4b506862d9d2bfb34'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:37.458193 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1438/2846] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:13:37 2022] DELETE /identity/v3/users/f087686a737847c4b506862d9d2bfb34 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:37.462817 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8762c5-2d6f-46f4-b685-dc86e2b664e3 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:37.463307 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8762c5-2d6f-46f4-b685-dc86e2b664e3 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:37.463408 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8762c5-2d6f-46f4-b685-dc86e2b664e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:37.481419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-5d8762c5-2d6f-46f4-b685-dc86e2b664e3 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:37.483249 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5d8762c5-2d6f-46f4-b685-dc86e2b664e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:37.540680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1409/2847] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:37.550875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:37.552638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:37.610141 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805340e3-0999-42d2-87cf-664db85d370b admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:37.613953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5637b6b-bf72-49a2-bdc4-5dc704c83d26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:37.614594 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5637b6b-bf72-49a2-bdc4-5dc704c83d26 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:37.614905 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5637b6b-bf72-49a2-bdc4-5dc704c83d26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:37.615182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5637b6b-bf72-49a2-bdc4-5dc704c83d26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:37.624441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5637b6b-bf72-49a2-bdc4-5dc704c83d26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:37.625891 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5637b6b-bf72-49a2-bdc4-5dc704c83d26 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:37.635138 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1439/2848] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:13:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:13:38.163610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8762c5-2d6f-46f4-b685-dc86e2b664e3 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:38.165113 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5d8762c5-2d6f-46f4-b685-dc86e2b664e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:38.178113 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8762c5-2d6f-46f4-b685-dc86e2b664e3 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:38.182631 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5976802-8597-43d7-bfe1-e351397df695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:38.183494 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5976802-8597-43d7-bfe1-e351397df695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:38.183774 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5976802-8597-43d7-bfe1-e351397df695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:38.184047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5976802-8597-43d7-bfe1-e351397df695 admin admin] PATH_INFO: `/v3/projects/09200991de3a43d68d209d275a3e08a1` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:38.185304 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5976802-8597-43d7-bfe1-e351397df695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09200991de3a43d68d209d275a3e08a1)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:38.187360 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5976802-8597-43d7-bfe1-e351397df695 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:38.235246 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a5976802-8597-43d7-bfe1-e351397df695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09200991de3a43d68d209d275a3e08a1', 'request_id': 'req-a5976802-8597-43d7-bfe1-e351397df695'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:38.240608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a5976802-8597-43d7-bfe1-e351397df695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09200991de3a43d68d209d275a3e08a1', 'request_id': 'req-a5976802-8597-43d7-bfe1-e351397df695'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:38.247092 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a5976802-8597-43d7-bfe1-e351397df695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09200991de3a43d68d209d275a3e08a1', 'request_id': 'req-a5976802-8597-43d7-bfe1-e351397df695'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:38.258058 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a5976802-8597-43d7-bfe1-e351397df695 admin admin] The token cache is being invalidate because project 09200991de3a43d68d209d275a3e08a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:13:38.258339 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a5976802-8597-43d7-bfe1-e351397df695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:38.258967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-a5976802-8597-43d7-bfe1-e351397df695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:13:38.260489 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1410/2849] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:13:38 2022] DELETE /identity/v3/projects/09200991de3a43d68d209d275a3e08a1 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:38.265632 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f729b-bea7-415b-9cb3-5f43bc4946b4 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:38.265927 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f729b-bea7-415b-9cb3-5f43bc4946b4 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:38.266211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f729b-bea7-415b-9cb3-5f43bc4946b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:38.283938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-fe7f729b-bea7-415b-9cb3-5f43bc4946b4 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:38.285183 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-fe7f729b-bea7-415b-9cb3-5f43bc4946b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:38.302367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:38.346317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1440/2850] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:38.354069 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a3bb2-dfcc-430a-aa39-a0206f94cd33 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:38.354359 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a3bb2-dfcc-430a-aa39-a0206f94cd33 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:38.354638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a3bb2-dfcc-430a-aa39-a0206f94cd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:38.373041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e78a3bb2-dfcc-430a-aa39-a0206f94cd33 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:38.374407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e78a3bb2-dfcc-430a-aa39-a0206f94cd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:38.382208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1411/2851] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:38.386390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f729b-bea7-415b-9cb3-5f43bc4946b4 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:38.396745 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f729b-bea7-415b-9cb3-5f43bc4946b4 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:38.401285 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc0254-53f5-46da-aadf-a017edb0c4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:38.401935 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc0254-53f5-46da-aadf-a017edb0c4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:38.402220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc0254-53f5-46da-aadf-a017edb0c4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:38.402497 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc0254-53f5-46da-aadf-a017edb0c4b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:38.403220 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc0254-53f5-46da-aadf-a017edb0c4b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:13:38.403675 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc0254-53f5-46da-aadf-a017edb0c4b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:38.405294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc0254-53f5-46da-aadf-a017edb0c4b0 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:38.416618 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1441/2852] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:13:38.421204 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a3bb2-dfcc-430a-aa39-a0206f94cd33 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:38.430201 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a3bb2-dfcc-430a-aa39-a0206f94cd33 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:38.435283 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f3d15-e5c8-48b7-ac62-33c84e3721ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:38.436013 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f3d15-e5c8-48b7-ac62-33c84e3721ce admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:38.436305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f3d15-e5c8-48b7-ac62-33c84e3721ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:38.436576 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f3d15-e5c8-48b7-ac62-33c84e3721ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:38.437288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f3d15-e5c8-48b7-ac62-33c84e3721ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:38.439179 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f3d15-e5c8-48b7-ac62-33c84e3721ce admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:38.474262 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1412/2853] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:13:38 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:13:38.478347 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc0254-53f5-46da-aadf-a017edb0c4b0 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:38.489652 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc0254-53f5-46da-aadf-a017edb0c4b0 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:38.546854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4fc91-1470-4e81-b6c3-6d2ac15e34a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:38.547530 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4fc91-1470-4e81-b6c3-6d2ac15e34a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:38.547817 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4fc91-1470-4e81-b6c3-6d2ac15e34a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:38.548093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4fc91-1470-4e81-b6c3-6d2ac15e34a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:38.548815 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4fc91-1470-4e81-b6c3-6d2ac15e34a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:38.550772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4fc91-1470-4e81-b6c3-6d2ac15e34a1 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:38.576565 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1442/2854] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:13:38 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:13:38.577768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f3d15-e5c8-48b7-ac62-33c84e3721ce admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:38.588035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f3d15-e5c8-48b7-ac62-33c84e3721ce admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:38.592809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af822715-5815-4de0-99fb-b2d49c999183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:38.593625 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af822715-5815-4de0-99fb-b2d49c999183 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:38.593913 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af822715-5815-4de0-99fb-b2d49c999183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:38.594192 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af822715-5815-4de0-99fb-b2d49c999183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:38.594909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af822715-5815-4de0-99fb-b2d49c999183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:38.596751 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af822715-5815-4de0-99fb-b2d49c999183 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:38.607798 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1413/2855] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:13:38.611507 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4fc91-1470-4e81-b6c3-6d2ac15e34a1 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:38.621468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4fc91-1470-4e81-b6c3-6d2ac15e34a1 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:38.627093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010b2a1-8fa6-4203-a62e-207e76bd523b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:38.627755 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010b2a1-8fa6-4203-a62e-207e76bd523b admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:38.628047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010b2a1-8fa6-4203-a62e-207e76bd523b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:38.628326 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010b2a1-8fa6-4203-a62e-207e76bd523b admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:38.629055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010b2a1-8fa6-4203-a62e-207e76bd523b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:38.630769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010b2a1-8fa6-4203-a62e-207e76bd523b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:38.641799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1443/2856] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:13:38.645882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af822715-5815-4de0-99fb-b2d49c999183 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:38.655999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af822715-5815-4de0-99fb-b2d49c999183 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:38.661133 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168ecac-6a98-46cf-a929-17587dc7d1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:38.662015 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168ecac-6a98-46cf-a929-17587dc7d1a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:38.662299 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168ecac-6a98-46cf-a929-17587dc7d1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:38.662575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168ecac-6a98-46cf-a929-17587dc7d1a1 admin admin] PATH_INFO: `/v3/projects/75a389b2c1ec4c78b499cc704f935116/users/c2028fd4d65d4d14a6b99343b1375463/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:38.677701 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168ecac-6a98-46cf-a929-17587dc7d1a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75a389b2c1ec4c78b499cc704f935116, user_id=c2028fd4d65d4d14a6b99343b1375463, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:38.680273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168ecac-6a98-46cf-a929-17587dc7d1a1 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:38.693872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1414/2857] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:13:38 2022] PUT /identity/v3/projects/75a389b2c1ec4c78b499cc704f935116/users/c2028fd4d65d4d14a6b99343b1375463/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:13:38.705210 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38744a4-dbb6-499a-a79b-bb5e18b35bec None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:38.705502 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38744a4-dbb6-499a-a79b-bb5e18b35bec None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:38.705781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38744a4-dbb6-499a-a79b-bb5e18b35bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:38.732512 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-b38744a4-dbb6-499a-a79b-bb5e18b35bec None None] MFA Rules not processed for user `c2028fd4d65d4d14a6b99343b1375463`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:13:38.733784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b38744a4-dbb6-499a-a79b-bb5e18b35bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:38.798706 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1444/2858] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:13:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:13:38.810016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168ecac-6a98-46cf-a929-17587dc7d1a1 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:13:38.812081 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4168ecac-6a98-46cf-a929-17587dc7d1a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:13:38.838786 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:38.872589 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168ecac-6a98-46cf-a929-17587dc7d1a1 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:13:38.876326 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c20c9-c9f0-4013-8efa-cc389154edad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:13:38.876969 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c20c9-c9f0-4013-8efa-cc389154edad service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:13:38.877290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c20c9-c9f0-4013-8efa-cc389154edad service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:13:38.877571 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c20c9-c9f0-4013-8efa-cc389154edad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:13:38.886083 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c20c9-c9f0-4013-8efa-cc389154edad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:13:38.887524 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c20c9-c9f0-4013-8efa-cc389154edad service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:13:38.897045 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1415/2859] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:13:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:13:48.795521 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:13:48.892836 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=17 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:14:15.760473 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38744a4-dbb6-499a-a79b-bb5e18b35bec None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:15.821737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38744a4-dbb6-499a-a79b-bb5e18b35bec None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:15.826820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:15.827680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:15.828057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:15.828336 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] PATH_INFO: `/v3/users/c2028fd4d65d4d14a6b99343b1375463` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:15.829534 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2028fd4d65d4d14a6b99343b1375463)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:15.831449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:15.876705 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2028fd4d65d4d14a6b99343b1375463'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:15.877367 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2028fd4d65d4d14a6b99343b1375463'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:15.884401 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2028fd4d65d4d14a6b99343b1375463'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:15.897605 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2028fd4d65d4d14a6b99343b1375463'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:15.898330 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2028fd4d65d4d14a6b99343b1375463'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:15.911317 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1445/2860] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:14:15 2022] DELETE /identity/v3/users/c2028fd4d65d4d14a6b99343b1375463 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:14:15.915756 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88a679-1d52-40e0-90c7-f9bc3ff66dbd None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:15.916311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88a679-1d52-40e0-90c7-f9bc3ff66dbd None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:15.916409 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88a679-1d52-40e0-90c7-f9bc3ff66dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:15.933989 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-1a88a679-1d52-40e0-90c7-f9bc3ff66dbd None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:14:15.936024 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1a88a679-1d52-40e0-90c7-f9bc3ff66dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:15.993488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1416/2861] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:14:16.003165 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:16.004658 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:16.061236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022994fc-57d0-4bfa-b1d9-38885e7bc5dc admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:16.064839 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83914e6-4cbc-4506-a079-a2b5f01dc51e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:16.065515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83914e6-4cbc-4506-a079-a2b5f01dc51e service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:16.065806 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83914e6-4cbc-4506-a079-a2b5f01dc51e service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:16.066081 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83914e6-4cbc-4506-a079-a2b5f01dc51e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:16.074972 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83914e6-4cbc-4506-a079-a2b5f01dc51e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:16.077047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83914e6-4cbc-4506-a079-a2b5f01dc51e service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:16.086389 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1446/2862] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:14:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:14:16.352106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88a679-1d52-40e0-90c7-f9bc3ff66dbd None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:16.353600 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-1a88a679-1d52-40e0-90c7-f9bc3ff66dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:16.367163 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88a679-1d52-40e0-90c7-f9bc3ff66dbd None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:16.372813 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f7e57-c302-4a01-bf88-39e83a98abf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:16.373707 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f7e57-c302-4a01-bf88-39e83a98abf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:16.373990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f7e57-c302-4a01-bf88-39e83a98abf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:16.374265 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f7e57-c302-4a01-bf88-39e83a98abf7 admin admin] PATH_INFO: `/v3/projects/75a389b2c1ec4c78b499cc704f935116` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:16.375536 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f7e57-c302-4a01-bf88-39e83a98abf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75a389b2c1ec4c78b499cc704f935116)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:16.378258 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f7e57-c302-4a01-bf88-39e83a98abf7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:16.427119 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-188f7e57-c302-4a01-bf88-39e83a98abf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75a389b2c1ec4c78b499cc704f935116', 'request_id': 'req-188f7e57-c302-4a01-bf88-39e83a98abf7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:16.432630 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-188f7e57-c302-4a01-bf88-39e83a98abf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75a389b2c1ec4c78b499cc704f935116', 'request_id': 'req-188f7e57-c302-4a01-bf88-39e83a98abf7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:16.440164 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-188f7e57-c302-4a01-bf88-39e83a98abf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75a389b2c1ec4c78b499cc704f935116', 'request_id': 'req-188f7e57-c302-4a01-bf88-39e83a98abf7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:16.451259 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-188f7e57-c302-4a01-bf88-39e83a98abf7 admin admin] The token cache is being invalidate because project 75a389b2c1ec4c78b499cc704f935116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:14:16.451543 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-188f7e57-c302-4a01-bf88-39e83a98abf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:16.452161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-188f7e57-c302-4a01-bf88-39e83a98abf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:16.453717 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1417/2863] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:14:16 2022] DELETE /identity/v3/projects/75a389b2c1ec4c78b499cc704f935116 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:14:16.458495 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302246ef-ad33-495d-b1ac-c85c7c1584be None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:16.458808 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302246ef-ad33-495d-b1ac-c85c7c1584be None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:16.459089 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302246ef-ad33-495d-b1ac-c85c7c1584be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:16.477465 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-302246ef-ad33-495d-b1ac-c85c7c1584be None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:14:16.478671 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-302246ef-ad33-495d-b1ac-c85c7c1584be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:16.496117 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:14:16.539963 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1447/2864] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:14:16.548014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e50289-3dcb-49a9-9344-a167a7ab3360 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:16.548309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e50289-3dcb-49a9-9344-a167a7ab3360 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:16.548591 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e50289-3dcb-49a9-9344-a167a7ab3360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:16.569101 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-c4e50289-3dcb-49a9-9344-a167a7ab3360 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:14:16.570312 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-c4e50289-3dcb-49a9-9344-a167a7ab3360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:16.579075 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1418/2865] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:14:16.583287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302246ef-ad33-495d-b1ac-c85c7c1584be None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:16.592892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302246ef-ad33-495d-b1ac-c85c7c1584be None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:16.597775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ddaca-6dc6-43cb-92ed-316eafd4ba6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:16.598427 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ddaca-6dc6-43cb-92ed-316eafd4ba6b admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:16.598714 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ddaca-6dc6-43cb-92ed-316eafd4ba6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:16.598988 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ddaca-6dc6-43cb-92ed-316eafd4ba6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:16.599701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ddaca-6dc6-43cb-92ed-316eafd4ba6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:14:16.600009 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ddaca-6dc6-43cb-92ed-316eafd4ba6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:16.601639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ddaca-6dc6-43cb-92ed-316eafd4ba6b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:16.613785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1448/2866] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:14:16.617809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e50289-3dcb-49a9-9344-a167a7ab3360 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:16.627447 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e50289-3dcb-49a9-9344-a167a7ab3360 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:16.632030 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25ceb9-11a5-423a-8b63-ff47b632c0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:16.632672 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25ceb9-11a5-423a-8b63-ff47b632c0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:16.632949 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25ceb9-11a5-423a-8b63-ff47b632c0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:16.633259 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25ceb9-11a5-423a-8b63-ff47b632c0f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:16.633934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25ceb9-11a5-423a-8b63-ff47b632c0f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:16.635792 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25ceb9-11a5-423a-8b63-ff47b632c0f0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:16.660829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1419/2867] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:14:16 2022] POST /identity/v3/projects => generated 397 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:14:16.664800 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ddaca-6dc6-43cb-92ed-316eafd4ba6b admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:16.675394 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ddaca-6dc6-43cb-92ed-316eafd4ba6b admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:16.732692 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d528a5b-393d-497c-80e1-db764666f88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:16.733388 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d528a5b-393d-497c-80e1-db764666f88f admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:16.733674 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d528a5b-393d-497c-80e1-db764666f88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:16.733953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d528a5b-393d-497c-80e1-db764666f88f admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:16.734641 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d528a5b-393d-497c-80e1-db764666f88f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:16.736581 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d528a5b-393d-497c-80e1-db764666f88f admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:16.756094 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1449/2868] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:14:16 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:14:16.760137 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25ceb9-11a5-423a-8b63-ff47b632c0f0 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:16.770440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25ceb9-11a5-423a-8b63-ff47b632c0f0 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:16.775011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc97152-0fcf-449d-ba4e-e9b4a08613e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:16.775662 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc97152-0fcf-449d-ba4e-e9b4a08613e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:16.775944 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc97152-0fcf-449d-ba4e-e9b4a08613e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:16.776221 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc97152-0fcf-449d-ba4e-e9b4a08613e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:16.776933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc97152-0fcf-449d-ba4e-e9b4a08613e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:16.778695 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc97152-0fcf-449d-ba4e-e9b4a08613e7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:16.789016 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1420/2869] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:14:16.792892 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d528a5b-393d-497c-80e1-db764666f88f admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:16.803598 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d528a5b-393d-497c-80e1-db764666f88f admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:16.808182 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07dcab3-0b06-4fe2-a6ea-8bbe4b08aae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:16.808832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dcab3-0b06-4fe2-a6ea-8bbe4b08aae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:16.809148 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dcab3-0b06-4fe2-a6ea-8bbe4b08aae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:16.809431 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dcab3-0b06-4fe2-a6ea-8bbe4b08aae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:16.810718 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07dcab3-0b06-4fe2-a6ea-8bbe4b08aae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:16.812294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07dcab3-0b06-4fe2-a6ea-8bbe4b08aae3 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:16.822133 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1450/2870] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:14:16.826547 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc97152-0fcf-449d-ba4e-e9b4a08613e7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:16.836354 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc97152-0fcf-449d-ba4e-e9b4a08613e7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:16.841019 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35124520-6a6e-4fe7-8d47-8a94594e50ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:16.841921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35124520-6a6e-4fe7-8d47-8a94594e50ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:16.842206 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35124520-6a6e-4fe7-8d47-8a94594e50ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:16.842484 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35124520-6a6e-4fe7-8d47-8a94594e50ea admin admin] PATH_INFO: `/v3/projects/73a6325f30fc4e9eb8a0faa71b8d86ed/users/9a0198638ba946919adeb441bf0bb08e/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:16.857837 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35124520-6a6e-4fe7-8d47-8a94594e50ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=73a6325f30fc4e9eb8a0faa71b8d86ed, user_id=9a0198638ba946919adeb441bf0bb08e, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:16.860424 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35124520-6a6e-4fe7-8d47-8a94594e50ea admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:16.873284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1421/2871] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:14:16 2022] PUT /identity/v3/projects/73a6325f30fc4e9eb8a0faa71b8d86ed/users/9a0198638ba946919adeb441bf0bb08e/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:14:16.884334 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e5fb0-9edf-475c-b9be-a11fd4ce6c6c None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:16.884631 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e5fb0-9edf-475c-b9be-a11fd4ce6c6c None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:16.884911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e5fb0-9edf-475c-b9be-a11fd4ce6c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:16.913499 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-f52e5fb0-9edf-475c-b9be-a11fd4ce6c6c None None] MFA Rules not processed for user `9a0198638ba946919adeb441bf0bb08e`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:14:16.914692 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f52e5fb0-9edf-475c-b9be-a11fd4ce6c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:16.977467 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1451/2872] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:14:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:14:16.989096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35124520-6a6e-4fe7-8d47-8a94594e50ea admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:16.990800 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-35124520-6a6e-4fe7-8d47-8a94594e50ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:17.013496 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:14:17.049819 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35124520-6a6e-4fe7-8d47-8a94594e50ea admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:17.053500 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f4ac7-9d49-4614-b29b-8734eb9da907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:17.054149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f4ac7-9d49-4614-b29b-8734eb9da907 service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:17.054434 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f4ac7-9d49-4614-b29b-8734eb9da907 service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:17.054712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f4ac7-9d49-4614-b29b-8734eb9da907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:17.063655 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f4ac7-9d49-4614-b29b-8734eb9da907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:17.065078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f4ac7-9d49-4614-b29b-8734eb9da907 service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:17.073951 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1422/2873] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:14:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:14:25.103907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e5fb0-9edf-475c-b9be-a11fd4ce6c6c None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:25.122638 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:14:25.167139 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e5fb0-9edf-475c-b9be-a11fd4ce6c6c None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:25.171731 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:25.172624 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:25.172911 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:25.173217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] PATH_INFO: `/v3/users/9a0198638ba946919adeb441bf0bb08e` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:25.174398 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a0198638ba946919adeb441bf0bb08e)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:25.176302 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:25.220769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a0198638ba946919adeb441bf0bb08e'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:25.221694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a0198638ba946919adeb441bf0bb08e'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:25.228402 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a0198638ba946919adeb441bf0bb08e'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:25.240260 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a0198638ba946919adeb441bf0bb08e'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:25.240928 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a0198638ba946919adeb441bf0bb08e'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:25.254140 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1452/2874] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:14:25 2022] DELETE /identity/v3/users/9a0198638ba946919adeb441bf0bb08e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:14:25.258684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862ec29-8d06-47f4-9de9-b112cca0a04f None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:25.259149 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862ec29-8d06-47f4-9de9-b112cca0a04f None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:25.259273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862ec29-8d06-47f4-9de9-b112cca0a04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:25.276920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-9862ec29-8d06-47f4-9de9-b112cca0a04f None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:14:25.278195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9862ec29-8d06-47f4-9de9-b112cca0a04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:25.336445 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1423/2875] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:14:25.346623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:25.348010 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:25.404435 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efacffb-f738-4af7-b858-1dba50ef5235 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:25.408162 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e6481-bada-4868-b604-8c40f63aabef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:25.408799 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e6481-bada-4868-b604-8c40f63aabef service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:25.409103 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e6481-bada-4868-b604-8c40f63aabef service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:25.409390 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e6481-bada-4868-b604-8c40f63aabef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:25.418346 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e6481-bada-4868-b604-8c40f63aabef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:25.419756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e6481-bada-4868-b604-8c40f63aabef service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:25.429106 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1453/2876] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:14:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:14:25.690807 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862ec29-8d06-47f4-9de9-b112cca0a04f None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:25.692316 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9862ec29-8d06-47f4-9de9-b112cca0a04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:25.705603 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862ec29-8d06-47f4-9de9-b112cca0a04f None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:25.710333 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db57ce7-33d6-4869-8b98-6176113b0776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:25.711200 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db57ce7-33d6-4869-8b98-6176113b0776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:25.711485 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db57ce7-33d6-4869-8b98-6176113b0776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:25.711765 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db57ce7-33d6-4869-8b98-6176113b0776 admin admin] PATH_INFO: `/v3/projects/73a6325f30fc4e9eb8a0faa71b8d86ed` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:25.713010 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db57ce7-33d6-4869-8b98-6176113b0776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73a6325f30fc4e9eb8a0faa71b8d86ed)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:25.714879 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db57ce7-33d6-4869-8b98-6176113b0776 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:25.763633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2db57ce7-33d6-4869-8b98-6176113b0776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73a6325f30fc4e9eb8a0faa71b8d86ed', 'request_id': 'req-2db57ce7-33d6-4869-8b98-6176113b0776'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:25.768955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2db57ce7-33d6-4869-8b98-6176113b0776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73a6325f30fc4e9eb8a0faa71b8d86ed', 'request_id': 'req-2db57ce7-33d6-4869-8b98-6176113b0776'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:25.775900 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2db57ce7-33d6-4869-8b98-6176113b0776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73a6325f30fc4e9eb8a0faa71b8d86ed', 'request_id': 'req-2db57ce7-33d6-4869-8b98-6176113b0776'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:25.786691 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2db57ce7-33d6-4869-8b98-6176113b0776 admin admin] The token cache is being invalidate because project 73a6325f30fc4e9eb8a0faa71b8d86ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:14:25.786971 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2db57ce7-33d6-4869-8b98-6176113b0776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:25.787587 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-2db57ce7-33d6-4869-8b98-6176113b0776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:14:25.789158 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1424/2877] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:14:25 2022] DELETE /identity/v3/projects/73a6325f30fc4e9eb8a0faa71b8d86ed => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:14:25.795464 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8d9f3-477f-472a-ac41-1fce2cfa30b9 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:25.795744 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8d9f3-477f-472a-ac41-1fce2cfa30b9 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:25.796025 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8d9f3-477f-472a-ac41-1fce2cfa30b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:25.814454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-eae8d9f3-477f-472a-ac41-1fce2cfa30b9 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:14:25.815663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-eae8d9f3-477f-472a-ac41-1fce2cfa30b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:25.848449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:14:25.875748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1454/2878] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:14:25.883575 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dec2b5-5552-48a1-8e6b-9cf0e2b5a1d8 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:25.883897 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dec2b5-5552-48a1-8e6b-9cf0e2b5a1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:25.884179 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dec2b5-5552-48a1-8e6b-9cf0e2b5a1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:25.904202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-75dec2b5-5552-48a1-8e6b-9cf0e2b5a1d8 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:14:25.905523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-75dec2b5-5552-48a1-8e6b-9cf0e2b5a1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:25.914660 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1425/2879] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:14:25.918878 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8d9f3-477f-472a-ac41-1fce2cfa30b9 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:25.928578 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8d9f3-477f-472a-ac41-1fce2cfa30b9 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:25.933150 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07405f89-7741-4f01-9892-4357d8076ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:25.933747 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07405f89-7741-4f01-9892-4357d8076ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:25.934030 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07405f89-7741-4f01-9892-4357d8076ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:25.934315 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07405f89-7741-4f01-9892-4357d8076ee7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:25.935154 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07405f89-7741-4f01-9892-4357d8076ee7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:14:25.935465 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07405f89-7741-4f01-9892-4357d8076ee7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:25.937060 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07405f89-7741-4f01-9892-4357d8076ee7 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:25.948346 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1455/2880] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:14:25.953037 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dec2b5-5552-48a1-8e6b-9cf0e2b5a1d8 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:25.962195 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dec2b5-5552-48a1-8e6b-9cf0e2b5a1d8 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:25.967610 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d5c89-e177-4a12-94d8-420c111be83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:25.968252 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d5c89-e177-4a12-94d8-420c111be83d admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:25.968533 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d5c89-e177-4a12-94d8-420c111be83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:25.968803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d5c89-e177-4a12-94d8-420c111be83d admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:25.969525 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d5c89-e177-4a12-94d8-420c111be83d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:25.971667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d5c89-e177-4a12-94d8-420c111be83d admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:25.996407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1426/2881] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:14:25 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 09:14:26.001384 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07405f89-7741-4f01-9892-4357d8076ee7 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:26.011622 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07405f89-7741-4f01-9892-4357d8076ee7 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:26.067981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde45e0-3c59-444d-b41b-3fc3245067a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:26.068647 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde45e0-3c59-444d-b41b-3fc3245067a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:26.068933 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde45e0-3c59-444d-b41b-3fc3245067a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:26.069251 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde45e0-3c59-444d-b41b-3fc3245067a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:26.069941 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde45e0-3c59-444d-b41b-3fc3245067a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:26.071848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde45e0-3c59-444d-b41b-3fc3245067a6 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:26.091758 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1456/2882] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:14:26 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:14:26.095934 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d5c89-e177-4a12-94d8-420c111be83d admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:26.107321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d5c89-e177-4a12-94d8-420c111be83d admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:26.111862 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d3ca1-3ca4-4674-ae24-c949d117f667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:26.112518 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d3ca1-3ca4-4674-ae24-c949d117f667 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:26.112801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d3ca1-3ca4-4674-ae24-c949d117f667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:26.113106 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d3ca1-3ca4-4674-ae24-c949d117f667 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:26.113822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d3ca1-3ca4-4674-ae24-c949d117f667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:26.115396 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d3ca1-3ca4-4674-ae24-c949d117f667 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:26.125678 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1427/2883] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:14:26.129636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde45e0-3c59-444d-b41b-3fc3245067a6 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:26.139760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde45e0-3c59-444d-b41b-3fc3245067a6 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:26.144417 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea68bd1-d907-4cf3-b63f-d02191954a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:26.145066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea68bd1-d907-4cf3-b63f-d02191954a7e admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:26.145385 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea68bd1-d907-4cf3-b63f-d02191954a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:26.145662 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea68bd1-d907-4cf3-b63f-d02191954a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:26.146381 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea68bd1-d907-4cf3-b63f-d02191954a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:26.147959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea68bd1-d907-4cf3-b63f-d02191954a7e admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:26.158142 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1457/2884] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:14:26.161970 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d3ca1-3ca4-4674-ae24-c949d117f667 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:26.172264 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d3ca1-3ca4-4674-ae24-c949d117f667 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:26.176906 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42311a-a896-4614-a18f-3b011b63b086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:26.177799 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42311a-a896-4614-a18f-3b011b63b086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:26.178081 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42311a-a896-4614-a18f-3b011b63b086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:26.178356 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42311a-a896-4614-a18f-3b011b63b086 admin admin] PATH_INFO: `/v3/projects/057cc9da8dc64caa89bf47f65fdd24b1/users/a5bc04ce186c40b0abfe4aa217cb1815/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:26.180551 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:14:26.195537 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42311a-a896-4614-a18f-3b011b63b086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=057cc9da8dc64caa89bf47f65fdd24b1, user_id=a5bc04ce186c40b0abfe4aa217cb1815, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:26.198166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42311a-a896-4614-a18f-3b011b63b086 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:26.210535 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1428/2885] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:14:26 2022] PUT /identity/v3/projects/057cc9da8dc64caa89bf47f65fdd24b1/users/a5bc04ce186c40b0abfe4aa217cb1815/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:14:26.215492 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f037e-bbdc-4f0a-8607-064dc2056d21 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:26.216327 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f037e-bbdc-4f0a-8607-064dc2056d21 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:26.216607 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f037e-bbdc-4f0a-8607-064dc2056d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:26.235093 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-7b3f037e-bbdc-4f0a-8607-064dc2056d21 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:14:26.236280 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-7b3f037e-bbdc-4f0a-8607-064dc2056d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:26.294755 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1458/2886] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:14:26.304263 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42311a-a896-4614-a18f-3b011b63b086 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:26.305990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-fc42311a-a896-4614-a18f-3b011b63b086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:26.365411 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42311a-a896-4614-a18f-3b011b63b086 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:26.369097 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f6a36-48a2-47e5-9105-d211b065869b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:26.369752 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f6a36-48a2-47e5-9105-d211b065869b service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:26.370038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f6a36-48a2-47e5-9105-d211b065869b service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:26.370319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f6a36-48a2-47e5-9105-d211b065869b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:26.379680 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f6a36-48a2-47e5-9105-d211b065869b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:26.381108 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f6a36-48a2-47e5-9105-d211b065869b service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:26.390980 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1429/2887] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:14:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:14:30.332447 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e15fe6-1e28-42ad-b245-6b86c3566c48 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:30.332962 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e15fe6-1e28-42ad-b245-6b86c3566c48 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:30.333045 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e15fe6-1e28-42ad-b245-6b86c3566c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:30.362523 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-03e15fe6-1e28-42ad-b245-6b86c3566c48 None None] MFA Rules not processed for user `a5bc04ce186c40b0abfe4aa217cb1815`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:14:30.363761 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-03e15fe6-1e28-42ad-b245-6b86c3566c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:30.400491 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:14:30.439311 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1459/2888] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:14:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:14:30.443730 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f6a36-48a2-47e5-9105-d211b065869b service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:30.459507 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f6a36-48a2-47e5-9105-d211b065869b service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:30.464944 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63fe63-528e-46ee-a2d7-2323a7656a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:30.465729 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63fe63-528e-46ee-a2d7-2323a7656a86 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:30.466193 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63fe63-528e-46ee-a2d7-2323a7656a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:30.466673 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63fe63-528e-46ee-a2d7-2323a7656a86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:30.467389 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63fe63-528e-46ee-a2d7-2323a7656a86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:30.469337 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63fe63-528e-46ee-a2d7-2323a7656a86 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:30.498112 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1430/2889] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:14:30 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:14:30.502032 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e15fe6-1e28-42ad-b245-6b86c3566c48 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:30.514221 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e15fe6-1e28-42ad-b245-6b86c3566c48 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:30.577826 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab89575-05fd-48ac-9fcc-b7349c1e282a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:30.578520 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab89575-05fd-48ac-9fcc-b7349c1e282a admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:30.578801 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab89575-05fd-48ac-9fcc-b7349c1e282a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:30.579072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab89575-05fd-48ac-9fcc-b7349c1e282a admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:30.579772 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab89575-05fd-48ac-9fcc-b7349c1e282a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:30.581736 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab89575-05fd-48ac-9fcc-b7349c1e282a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:30.605827 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1460/2890] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:14:30 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:14:30.609762 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63fe63-528e-46ee-a2d7-2323a7656a86 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:30.620867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63fe63-528e-46ee-a2d7-2323a7656a86 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:30.626407 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36876ae6-a541-46da-b864-f36da13dc685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:30.627086 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36876ae6-a541-46da-b864-f36da13dc685 admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:30.627373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36876ae6-a541-46da-b864-f36da13dc685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:30.627649 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36876ae6-a541-46da-b864-f36da13dc685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:30.628373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36876ae6-a541-46da-b864-f36da13dc685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:30.630017 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36876ae6-a541-46da-b864-f36da13dc685 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:30.642140 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1431/2891] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:14:30.646084 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab89575-05fd-48ac-9fcc-b7349c1e282a admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:30.656934 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab89575-05fd-48ac-9fcc-b7349c1e282a admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:30.661717 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec29a3b-8124-45b8-8f09-ad3e7b5b1d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:30.662609 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec29a3b-8124-45b8-8f09-ad3e7b5b1d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:30.662898 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec29a3b-8124-45b8-8f09-ad3e7b5b1d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:30.663175 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec29a3b-8124-45b8-8f09-ad3e7b5b1d02 admin admin] PATH_INFO: `/v3/projects/3dfb813e45574e3e927d6f6e27713220/users/45532fa2cf1a40aebfae048b7943a42c/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:30.678677 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec29a3b-8124-45b8-8f09-ad3e7b5b1d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfb813e45574e3e927d6f6e27713220, user_id=45532fa2cf1a40aebfae048b7943a42c, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:30.681436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec29a3b-8124-45b8-8f09-ad3e7b5b1d02 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:30.696831 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1461/2892] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:14:30 2022] PUT /identity/v3/projects/3dfb813e45574e3e927d6f6e27713220/users/45532fa2cf1a40aebfae048b7943a42c/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:14:30.701561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36876ae6-a541-46da-b864-f36da13dc685 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:30.712649 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36876ae6-a541-46da-b864-f36da13dc685 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:30.782611 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda51bb2-881b-4d4c-9359-f9c46214d55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:30.783291 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda51bb2-881b-4d4c-9359-f9c46214d55e admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:30.783573 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda51bb2-881b-4d4c-9359-f9c46214d55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:30.783848 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda51bb2-881b-4d4c-9359-f9c46214d55e admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:30.784576 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda51bb2-881b-4d4c-9359-f9c46214d55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:30.786193 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda51bb2-881b-4d4c-9359-f9c46214d55e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:30.796770 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1432/2893] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:14:30.800882 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec29a3b-8124-45b8-8f09-ad3e7b5b1d02 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:30.811951 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec29a3b-8124-45b8-8f09-ad3e7b5b1d02 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:30.816656 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51b3a8-fbd1-4750-8356-b1d9f70cdcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:30.817459 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51b3a8-fbd1-4750-8356-b1d9f70cdcbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:30.817741 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51b3a8-fbd1-4750-8356-b1d9f70cdcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:30.818013 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51b3a8-fbd1-4750-8356-b1d9f70cdcbe admin admin] PATH_INFO: `/v3/projects/3dfb813e45574e3e927d6f6e27713220/users/45532fa2cf1a40aebfae048b7943a42c/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:30.819953 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51b3a8-fbd1-4750-8356-b1d9f70cdcbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfb813e45574e3e927d6f6e27713220, user_id=45532fa2cf1a40aebfae048b7943a42c, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:30.822613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51b3a8-fbd1-4750-8356-b1d9f70cdcbe admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:30.834043 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1462/2894] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:14:30 2022] PUT /identity/v3/projects/3dfb813e45574e3e927d6f6e27713220/users/45532fa2cf1a40aebfae048b7943a42c/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:14:35.058432 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c10e1-7116-4cd2-8c69-cf90330943ac None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:35.058955 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c10e1-7116-4cd2-8c69-cf90330943ac None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:35.059469 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c10e1-7116-4cd2-8c69-cf90330943ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:35.088618 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-f49c10e1-7116-4cd2-8c69-cf90330943ac None None] MFA Rules not processed for user `45532fa2cf1a40aebfae048b7943a42c`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:14:35.089899 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-f49c10e1-7116-4cd2-8c69-cf90330943ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:35.132966 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:14:35.164877 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1433/2895] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:14:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:14:35.176926 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51b3a8-fbd1-4750-8356-b1d9f70cdcbe admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:35.178711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6c51b3a8-fbd1-4750-8356-b1d9f70cdcbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:35.261441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51b3a8-fbd1-4750-8356-b1d9f70cdcbe admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:35.266072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bc873-b1d0-4c34-abdb-38fa13fc71d1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e4531ac696445ba5bfa9870984befd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:35.266726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bc873-b1d0-4c34-abdb-38fa13fc71d1 service glance] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:35.267008 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bc873-b1d0-4c34-abdb-38fa13fc71d1 service glance] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:35.267284 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bc873-b1d0-4c34-abdb-38fa13fc71d1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:35.290444 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:14:35.329823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bc873-b1d0-4c34-abdb-38fa13fc71d1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:35.331553 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bc873-b1d0-4c34-abdb-38fa13fc71d1 service glance] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:35.341481 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1463/2896] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:14:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:14:35.447344 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ed36f-d1a5-4b26-ac73-a43dca4351cf None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:35.447710 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ed36f-d1a5-4b26-ac73-a43dca4351cf None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:35.448076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ed36f-d1a5-4b26-ac73-a43dca4351cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:35.466768 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-4a5ed36f-d1a5-4b26-ac73-a43dca4351cf None None] MFA Rules not processed for user `26d6eb06b23b4db88261ae58d650736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:14:35.468011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-4a5ed36f-d1a5-4b26-ac73-a43dca4351cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:35.526523 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1434/2897] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jun 7 09:14:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:14:35.576029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bc873-b1d0-4c34-abdb-38fa13fc71d1 service glance] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:35.577560 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-476bc873-b1d0-4c34-abdb-38fa13fc71d1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:35.653623 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bc873-b1d0-4c34-abdb-38fa13fc71d1 service glance] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:35.658769 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b4882-50f1-422c-ad57-f0a4c3d6d3cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586ddffa77a54521b42b2aa50f16cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:35.659420 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b4882-50f1-422c-ad57-f0a4c3d6d3cc service swift] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:35.659707 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b4882-50f1-422c-ad57-f0a4c3d6d3cc service swift] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:35.659981 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b4882-50f1-422c-ad57-f0a4c3d6d3cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:35.668636 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b4882-50f1-422c-ad57-f0a4c3d6d3cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:35.670261 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b4882-50f1-422c-ad57-f0a4c3d6d3cc service swift] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:35.678770 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1464/2898] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jun 7 09:14:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:14:44.411076 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5465643-aa9a-4521-8cd1-f27f84f535d6 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:44.411687 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5465643-aa9a-4521-8cd1-f27f84f535d6 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:44.411813 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5465643-aa9a-4521-8cd1-f27f84f535d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:44.429975 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-e5465643-aa9a-4521-8cd1-f27f84f535d6 None None] MFA Rules not processed for user `a5bc04ce186c40b0abfe4aa217cb1815`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:14:44.431171 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-e5465643-aa9a-4521-8cd1-f27f84f535d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:44.439561 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1435/2899] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:14:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:14:45.675351 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=49 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:14:50.241137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2e8e9-a61c-40af-a8d5-c3bc4d50b488 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:50.241756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2e8e9-a61c-40af-a8d5-c3bc4d50b488 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:50.241756 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2e8e9-a61c-40af-a8d5-c3bc4d50b488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:50.260323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-f4d2e8e9-a61c-40af-a8d5-c3bc4d50b488 None None] MFA Rules not processed for user `26d6eb06b23b4db88261ae58d650736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:14:50.261829 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f4d2e8e9-a61c-40af-a8d5-c3bc4d50b488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:14:50.269566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1465/2900] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jun 7 09:14:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:14:50.283789 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5465643-aa9a-4521-8cd1-f27f84f535d6 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:14:50.297060 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5465643-aa9a-4521-8cd1-f27f84f535d6 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:14:50.302202 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1634e7-f169-44ba-9d9b-0d3553cca1f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586ddffa77a54521b42b2aa50f16cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:14:50.302963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1634e7-f169-44ba-9d9b-0d3553cca1f3 service swift] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:14:50.303321 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1634e7-f169-44ba-9d9b-0d3553cca1f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:14:50.303704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1634e7-f169-44ba-9d9b-0d3553cca1f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:14:50.313319 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1634e7-f169-44ba-9d9b-0d3553cca1f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:14:50.314909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1634e7-f169-44ba-9d9b-0d3553cca1f3 service swift] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:14:50.323489 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1436/2901] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jun 7 09:14:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:15:00.255333 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:15:00.319933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=46 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:15:04.430723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae579f8a-c568-459b-8ad2-6187a8679359 None None] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:15:04.431264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae579f8a-c568-459b-8ad2-6187a8679359 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:15:04.431264 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae579f8a-c568-459b-8ad2-6187a8679359 None None] PATH_INFO: `/` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:15:04.432663 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1466/2902] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jun 7 09:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:15:04.439413 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23378107-2aaa-4cde-ad8f-3f77a22548de None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:15:04.439896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23378107-2aaa-4cde-ad8f-3f77a22548de None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:15:04.440135 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23378107-2aaa-4cde-ad8f-3f77a22548de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:15:04.490051 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-23378107-2aaa-4cde-ad8f-3f77a22548de None None] MFA Rules not processed for user `7ddf89deb28749eda2d26ad66bc282f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:15:04.491166 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-23378107-2aaa-4cde-ad8f-3f77a22548de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:15:04.545309 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1437/2903] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jun 7 09:15:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 09:15:04.556820 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae579f8a-c568-459b-8ad2-6187a8679359 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:15:04.569535 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae579f8a-c568-459b-8ad2-6187a8679359 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:15:04.573016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa2067-2e7d-4f58-9696-2e8b0457c056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:15:04.573726 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2067-2e7d-4f58-9696-2e8b0457c056 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:15:04.574019 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2067-2e7d-4f58-9696-2e8b0457c056 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:15:04.574289 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2067-2e7d-4f58-9696-2e8b0457c056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:15:04.582016 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa2067-2e7d-4f58-9696-2e8b0457c056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:15:04.583436 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa2067-2e7d-4f58-9696-2e8b0457c056 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:15:04.591096 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1467/2904] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:15:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:15:04.671255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df087f2-b8e4-4415-ae80-b7d16c68e4b9 None None] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:15:04.671549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df087f2-b8e4-4415-ae80-b7d16c68e4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:15:04.671829 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df087f2-b8e4-4415-ae80-b7d16c68e4b9 None None] PATH_INFO: `/` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:15:04.673241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1438/2905] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jun 7 09:15:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 09:15:04.678441 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81192-1d57-4005-9f60-17b256f80fdc None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:15:04.678732 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81192-1d57-4005-9f60-17b256f80fdc None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:15:04.679012 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81192-1d57-4005-9f60-17b256f80fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:15:04.719684 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-71d81192-1d57-4005-9f60-17b256f80fdc None None] MFA Rules not processed for user `3f2a2bf913d1412f996acf4401ba7673`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:15:04.720915 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-71d81192-1d57-4005-9f60-17b256f80fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:15:04.791290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1468/2906] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jun 7 09:15:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:15:04.803961 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df087f2-b8e4-4415-ae80-b7d16c68e4b9 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:15:04.805725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-6df087f2-b8e4-4415-ae80-b7d16c68e4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:15:04.819557 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df087f2-b8e4-4415-ae80-b7d16c68e4b9 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:15:04.823113 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5917d-8522-432e-8ec9-9e0d9d8b6056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:15:04.823753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5917d-8522-432e-8ec9-9e0d9d8b6056 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:15:04.824038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5917d-8522-432e-8ec9-9e0d9d8b6056 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:15:04.824323 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5917d-8522-432e-8ec9-9e0d9d8b6056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:15:04.834787 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5917d-8522-432e-8ec9-9e0d9d8b6056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:15:04.836241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5917d-8522-432e-8ec9-9e0d9d8b6056 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:15:04.847721 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1439/2907] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:15:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:15:14.781479 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=43 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:15:14.842436 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=35 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:15:20.901859 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d81192-1d57-4005-9f60-17b256f80fdc None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:15:20.911883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d81192-1d57-4005-9f60-17b256f80fdc None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:15:20.915416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5118da6-f5fe-4bc0-9a56-df5115293a68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:15:20.916055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5118da6-f5fe-4bc0-9a56-df5115293a68 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:15:20.916339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5118da6-f5fe-4bc0-9a56-df5115293a68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:15:20.916615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5118da6-f5fe-4bc0-9a56-df5115293a68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:15:20.926368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5118da6-f5fe-4bc0-9a56-df5115293a68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:15:20.927783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5118da6-f5fe-4bc0-9a56-df5115293a68 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:15:20.937121 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1469/2908] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:15:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:15:30.932999 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:15:57.791517 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5917d-8522-432e-8ec9-9e0d9d8b6056 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:15:57.800384 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5917d-8522-432e-8ec9-9e0d9d8b6056 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:15:57.804070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5606a-8541-4fc8-9dcb-06b69fd0ee9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:15:57.804704 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5606a-8541-4fc8-9dcb-06b69fd0ee9e service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:15:57.804992 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5606a-8541-4fc8-9dcb-06b69fd0ee9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:15:57.805305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5606a-8541-4fc8-9dcb-06b69fd0ee9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:15:57.807073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-39c5606a-8541-4fc8-9dcb-06b69fd0ee9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:15:57.820748 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5606a-8541-4fc8-9dcb-06b69fd0ee9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:15:57.822190 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5606a-8541-4fc8-9dcb-06b69fd0ee9e service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:15:57.833871 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1440/2909] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:15:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:16:07.828219 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:16:29.044835 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5118da6-f5fe-4bc0-9a56-df5115293a68 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:29.047062 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-d5118da6-f5fe-4bc0-9a56-df5115293a68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:29.062828 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5118da6-f5fe-4bc0-9a56-df5115293a68 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:29.068442 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a8055-10d8-4c00-ba84-60dfb4fa9222 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:29.069137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a8055-10d8-4c00-ba84-60dfb4fa9222 service nova] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:29.069416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a8055-10d8-4c00-ba84-60dfb4fa9222 service nova] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:29.069699 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a8055-10d8-4c00-ba84-60dfb4fa9222 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:29.071454 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c69a8055-10d8-4c00-ba84-60dfb4fa9222 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:29.083816 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a8055-10d8-4c00-ba84-60dfb4fa9222 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:29.085966 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a8055-10d8-4c00-ba84-60dfb4fa9222 service nova] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:29.096621 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1470/2910] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:16:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:16:35.883866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5606a-8541-4fc8-9dcb-06b69fd0ee9e service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:35.887011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-39c5606a-8541-4fc8-9dcb-06b69fd0ee9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:35.983990 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5606a-8541-4fc8-9dcb-06b69fd0ee9e service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:35.989499 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b5d5e-ca40-4b92-95ea-34931c14a881 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:35.990921 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b5d5e-ca40-4b92-95ea-34931c14a881 service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:35.991602 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b5d5e-ca40-4b92-95ea-34931c14a881 service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:35.992684 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b5d5e-ca40-4b92-95ea-34931c14a881 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:35.994794 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-223b5d5e-ca40-4b92-95ea-34931c14a881 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:36.015187 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b5d5e-ca40-4b92-95ea-34931c14a881 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:36.016920 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b5d5e-ca40-4b92-95ea-34931c14a881 service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:36.031497 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1441/2911] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:16:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:16:39.092098 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:16:39.205785 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a8055-10d8-4c00-ba84-60dfb4fa9222 service nova] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:39.207290 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c69a8055-10d8-4c00-ba84-60dfb4fa9222 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:39.272889 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a8055-10d8-4c00-ba84-60dfb4fa9222 service nova] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:39.277509 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:39.278382 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:39.278669 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:39.278946 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] PATH_INFO: `/v3/users/4cb845105733481e9b3f32d3c8c2c402` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:39.280122 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cb845105733481e9b3f32d3c8c2c402)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:39.282090 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:39.328104 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cb845105733481e9b3f32d3c8c2c402'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.328773 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cb845105733481e9b3f32d3c8c2c402'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.342066 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cb845105733481e9b3f32d3c8c2c402'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.361994 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cb845105733481e9b3f32d3c8c2c402'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.365513 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cb845105733481e9b3f32d3c8c2c402'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.380023 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1471/2912] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:16:39 2022] DELETE /identity/v3/users/4cb845105733481e9b3f32d3c8c2c402 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:16:39.428604 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c8a00-2da0-4863-b302-760f4bcf07ab None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:39.429062 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c8a00-2da0-4863-b302-760f4bcf07ab None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:39.429208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c8a00-2da0-4863-b302-760f4bcf07ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:39.448633 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-b62c8a00-2da0-4863-b302-760f4bcf07ab None None] MFA Rules not processed for user `26d6eb06b23b4db88261ae58d650736d`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:16:39.449866 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b62c8a00-2da0-4863-b302-760f4bcf07ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:39.507398 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1442/2913] 10.222.0.39 () {58 vars in 967 bytes} [Tue Jun 7 09:16:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:16:39.555120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:39.556469 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:39.559570 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c8a00-2da0-4863-b302-760f4bcf07ab None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:39.561073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b62c8a00-2da0-4863-b302-760f4bcf07ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:39.638748 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2eac4-2545-4dd2-bad0-0b756b39b932 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:39.645146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c8a00-2da0-4863-b302-760f4bcf07ab None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:39.645639 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5329f-28b8-4efc-84c8-4f4bc58b55f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586ddffa77a54521b42b2aa50f16cb92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:39.647028 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5329f-28b8-4efc-84c8-4f4bc58b55f6 service swift] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:39.647400 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5329f-28b8-4efc-84c8-4f4bc58b55f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:39.647795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5329f-28b8-4efc-84c8-4f4bc58b55f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:39.651069 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:39.652165 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:39.652448 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:39.652726 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] PATH_INFO: `/v3/projects/a05df95bc7b444e494aa98b09f6f3e52` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:39.654160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a05df95bc7b444e494aa98b09f6f3e52)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:39.656788 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5329f-28b8-4efc-84c8-4f4bc58b55f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:39.656928 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:39.658419 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5329f-28b8-4efc-84c8-4f4bc58b55f6 service swift] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:39.667666 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1472/2914] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jun 7 09:16:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 09:16:39.700035 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:16:39.712742 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a05df95bc7b444e494aa98b09f6f3e52', 'request_id': 'req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.718559 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a05df95bc7b444e494aa98b09f6f3e52', 'request_id': 'req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.727963 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a05df95bc7b444e494aa98b09f6f3e52', 'request_id': 'req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.742063 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] The token cache is being invalidate because project a05df95bc7b444e494aa98b09f6f3e52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:16:39.742362 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.743088 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.744925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1443/2915] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:16:39 2022] DELETE /identity/v3/projects/a05df95bc7b444e494aa98b09f6f3e52 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:16:39.748695 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5329f-28b8-4efc-84c8-4f4bc58b55f6 service swift] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:39.760548 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5329f-28b8-4efc-84c8-4f4bc58b55f6 service swift] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:39.765438 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:16:39.829947 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92996f15-4330-4b00-a206-9c362997a31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:39.830830 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92996f15-4330-4b00-a206-9c362997a31d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:39.831118 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92996f15-4330-4b00-a206-9c362997a31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:39.831397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92996f15-4330-4b00-a206-9c362997a31d admin admin] PATH_INFO: `/v3/users/7f78151ceafd4dfab3dca80123671468` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:39.832612 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92996f15-4330-4b00-a206-9c362997a31d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f78151ceafd4dfab3dca80123671468)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:39.834633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92996f15-4330-4b00-a206-9c362997a31d admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:39.885615 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-92996f15-4330-4b00-a206-9c362997a31d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f78151ceafd4dfab3dca80123671468'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.886323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-92996f15-4330-4b00-a206-9c362997a31d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f78151ceafd4dfab3dca80123671468'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.894217 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-92996f15-4330-4b00-a206-9c362997a31d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f78151ceafd4dfab3dca80123671468'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.909977 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-92996f15-4330-4b00-a206-9c362997a31d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f78151ceafd4dfab3dca80123671468'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.910717 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-92996f15-4330-4b00-a206-9c362997a31d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f78151ceafd4dfab3dca80123671468'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:39.925025 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1473/2916] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:16:39 2022] DELETE /identity/v3/users/7f78151ceafd4dfab3dca80123671468 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:16:40.114342 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:40.116692 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:40.133842 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f57d9-4899-4567-8f62-5a2d68b59fc2 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:40.202133 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8dbcf8-155e-451d-956e-4080aaac6e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:40.203682 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8dbcf8-155e-451d-956e-4080aaac6e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:40.204038 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8dbcf8-155e-451d-956e-4080aaac6e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:40.204328 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8dbcf8-155e-451d-956e-4080aaac6e01 admin admin] PATH_INFO: `/v3/projects/55e04881ccc34a06a8d235ad4aa28f33` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:40.206965 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8dbcf8-155e-451d-956e-4080aaac6e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e04881ccc34a06a8d235ad4aa28f33)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:40.209639 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8dbcf8-155e-451d-956e-4080aaac6e01 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:40.265314 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cf8dbcf8-155e-451d-956e-4080aaac6e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e04881ccc34a06a8d235ad4aa28f33', 'request_id': 'req-cf8dbcf8-155e-451d-956e-4080aaac6e01'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:40.271355 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cf8dbcf8-155e-451d-956e-4080aaac6e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e04881ccc34a06a8d235ad4aa28f33', 'request_id': 'req-cf8dbcf8-155e-451d-956e-4080aaac6e01'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:40.279964 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cf8dbcf8-155e-451d-956e-4080aaac6e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e04881ccc34a06a8d235ad4aa28f33', 'request_id': 'req-cf8dbcf8-155e-451d-956e-4080aaac6e01'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:40.292235 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cf8dbcf8-155e-451d-956e-4080aaac6e01 admin admin] The token cache is being invalidate because project 55e04881ccc34a06a8d235ad4aa28f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:16:40.292527 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cf8dbcf8-155e-451d-956e-4080aaac6e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:40.293215 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-cf8dbcf8-155e-451d-956e-4080aaac6e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:40.294788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1444/2917] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:16:40 2022] DELETE /identity/v3/projects/55e04881ccc34a06a8d235ad4aa28f33 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:16:40.299802 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85797fd3-2a1a-4758-bf99-1aad0ffdc9a4 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:40.300287 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85797fd3-2a1a-4758-bf99-1aad0ffdc9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:40.300379 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85797fd3-2a1a-4758-bf99-1aad0ffdc9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:40.319234 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-85797fd3-2a1a-4758-bf99-1aad0ffdc9a4 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:16:40.320532 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-85797fd3-2a1a-4758-bf99-1aad0ffdc9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:40.514822 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1474/2918] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:16:40.523241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa111a-2767-4631-b322-b08369c47968 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:40.523759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa111a-2767-4631-b322-b08369c47968 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:40.523851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa111a-2767-4631-b322-b08369c47968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:40.542453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-16aa111a-2767-4631-b322-b08369c47968 None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:16:40.543857 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-16aa111a-2767-4631-b322-b08369c47968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:40.551616 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1445/2919] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:16:40.555942 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85797fd3-2a1a-4758-bf99-1aad0ffdc9a4 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:40.565300 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85797fd3-2a1a-4758-bf99-1aad0ffdc9a4 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:40.570848 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaefbf-2b0f-4577-98b5-4ada3aa3367b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:40.571504 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaefbf-2b0f-4577-98b5-4ada3aa3367b admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:40.571784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaefbf-2b0f-4577-98b5-4ada3aa3367b admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:40.572056 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaefbf-2b0f-4577-98b5-4ada3aa3367b admin admin] PATH_INFO: `/v3/domains` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:40.572784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aaefbf-2b0f-4577-98b5-4ada3aa3367b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80229) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 09:16:40.573112 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aaefbf-2b0f-4577-98b5-4ada3aa3367b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:40.574781 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aaefbf-2b0f-4577-98b5-4ada3aa3367b admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:40.587635 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1475/2920] 10.222.0.39 () {58 vars in 1152 bytes} [Tue Jun 7 09:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 09:16:40.592071 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa111a-2767-4631-b322-b08369c47968 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:40.602552 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa111a-2767-4631-b322-b08369c47968 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:40.607580 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95709a-a0c4-4143-8c21-dd212c422c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:40.608240 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95709a-a0c4-4143-8c21-dd212c422c5b admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:40.608522 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95709a-a0c4-4143-8c21-dd212c422c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:40.608798 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95709a-a0c4-4143-8c21-dd212c422c5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:40.609518 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95709a-a0c4-4143-8c21-dd212c422c5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:40.611608 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95709a-a0c4-4143-8c21-dd212c422c5b admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:40.642681 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1446/2921] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:16:40 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:16:40.643985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaefbf-2b0f-4577-98b5-4ada3aa3367b admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:40.655166 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaefbf-2b0f-4577-98b5-4ada3aa3367b admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:40.668589 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:16:40.718251 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcf24e-03c6-44c0-9ce8-077849c52e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:40.718924 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcf24e-03c6-44c0-9ce8-077849c52e31 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:40.719209 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcf24e-03c6-44c0-9ce8-077849c52e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:40.719517 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcf24e-03c6-44c0-9ce8-077849c52e31 admin admin] PATH_INFO: `/v3/users` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:40.720205 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcf24e-03c6-44c0-9ce8-077849c52e31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:40.722442 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcf24e-03c6-44c0-9ce8-077849c52e31 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:40.743253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1476/2922] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:16:40 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:16:40.747323 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95709a-a0c4-4143-8c21-dd212c422c5b admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:40.758305 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95709a-a0c4-4143-8c21-dd212c422c5b admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:40.763059 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63fdf1-565d-4292-b928-b6ae0d36908e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:40.763725 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63fdf1-565d-4292-b928-b6ae0d36908e admin admin] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:40.764012 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63fdf1-565d-4292-b928-b6ae0d36908e admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:40.764290 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63fdf1-565d-4292-b928-b6ae0d36908e admin admin] PATH_INFO: `/v3/roles` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:40.765011 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63fdf1-565d-4292-b928-b6ae0d36908e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:40.766646 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63fdf1-565d-4292-b928-b6ae0d36908e admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:40.777777 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1447/2923] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:16:40.781611 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcf24e-03c6-44c0-9ce8-077849c52e31 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:40.792440 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcf24e-03c6-44c0-9ce8-077849c52e31 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:40.797192 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a393b-f099-4da5-bc8f-063ce621a250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:40.797798 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a393b-f099-4da5-bc8f-063ce621a250 admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:40.798083 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a393b-f099-4da5-bc8f-063ce621a250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:40.798524 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a393b-f099-4da5-bc8f-063ce621a250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:40.799253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a393b-f099-4da5-bc8f-063ce621a250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:40.800842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a393b-f099-4da5-bc8f-063ce621a250 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:40.811465 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1477/2924] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:16:40.815461 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63fdf1-565d-4292-b928-b6ae0d36908e admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:40.826150 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63fdf1-565d-4292-b928-b6ae0d36908e admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:40.830839 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dac3c2-bf55-4f55-8be5-91c1e27fe500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:40.831716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dac3c2-bf55-4f55-8be5-91c1e27fe500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:40.831999 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dac3c2-bf55-4f55-8be5-91c1e27fe500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:40.832276 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dac3c2-bf55-4f55-8be5-91c1e27fe500 admin admin] PATH_INFO: `/v3/projects/de4176cee52a461fa7eeb6a471e82593/users/97dfa9023e15415d85b6f4650804d6d9/roles/2bdabfafa05e443e842fe34d5f03e2c9` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:40.845176 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:16:40.849719 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dac3c2-bf55-4f55-8be5-91c1e27fe500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de4176cee52a461fa7eeb6a471e82593, user_id=97dfa9023e15415d85b6f4650804d6d9, role_id=2bdabfafa05e443e842fe34d5f03e2c9)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:40.852793 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dac3c2-bf55-4f55-8be5-91c1e27fe500 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:40.868220 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1448/2925] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:16:40 2022] PUT /identity/v3/projects/de4176cee52a461fa7eeb6a471e82593/users/97dfa9023e15415d85b6f4650804d6d9/roles/2bdabfafa05e443e842fe34d5f03e2c9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:16:40.873294 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbd06b-42ce-4d5a-8272-1c0f92980f2b None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:40.873681 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbd06b-42ce-4d5a-8272-1c0f92980f2b None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:40.874029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbd06b-42ce-4d5a-8272-1c0f92980f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:40.898050 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-11bbd06b-42ce-4d5a-8272-1c0f92980f2b None None] MFA Rules not processed for user `d70de4148c8f4696ac913ae91f5a81cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:16:40.899397 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-11bbd06b-42ce-4d5a-8272-1c0f92980f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:40.968057 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1478/2926] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:16:40.978104 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dac3c2-bf55-4f55-8be5-91c1e27fe500 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:40.979851 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-99dac3c2-bf55-4f55-8be5-91c1e27fe500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:41.051096 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dac3c2-bf55-4f55-8be5-91c1e27fe500 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:41.055933 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6161b94-57ee-4b5a-9f19-ae0a05958bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:41.056662 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6161b94-57ee-4b5a-9f19-ae0a05958bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:41.056949 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6161b94-57ee-4b5a-9f19-ae0a05958bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:41.057259 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6161b94-57ee-4b5a-9f19-ae0a05958bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:41.067556 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6161b94-57ee-4b5a-9f19-ae0a05958bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:41.069264 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6161b94-57ee-4b5a-9f19-ae0a05958bb1 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:41.080451 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1449/2927] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:16:46.504548 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbd06b-42ce-4d5a-8272-1c0f92980f2b None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:46.506175 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-11bbd06b-42ce-4d5a-8272-1c0f92980f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:46.525780 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a762c-72b2-4545-a2b1-279bae5b4443 None None] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:46.526311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a762c-72b2-4545-a2b1-279bae5b4443 None None] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:46.526311 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a762c-72b2-4545-a2b1-279bae5b4443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:46.591724 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.auth.core [None req-577a762c-72b2-4545-a2b1-279bae5b4443 None None] MFA Rules not processed for user `97dfa9023e15415d85b6f4650804d6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80228) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:16:46.595349 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-577a762c-72b2-4545-a2b1-279bae5b4443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:46.661340 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbd06b-42ce-4d5a-8272-1c0f92980f2b None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:46.671000 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c041f-374b-4ef0-8da0-1593d17ff147 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:46.671672 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c041f-374b-4ef0-8da0-1593d17ff147 service placement] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:46.671959 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c041f-374b-4ef0-8da0-1593d17ff147 service placement] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:46.672236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c041f-374b-4ef0-8da0-1593d17ff147 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:46.675683 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-ac3c041f-374b-4ef0-8da0-1593d17ff147 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:46.692179 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c041f-374b-4ef0-8da0-1593d17ff147 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:46.695052 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c041f-374b-4ef0-8da0-1593d17ff147 service placement] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:46.707832 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1479/2928] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:16:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:16:46.754399 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1450/2929] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:16:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:16:46.760823 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c041f-374b-4ef0-8da0-1593d17ff147 service placement] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:46.777259 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c041f-374b-4ef0-8da0-1593d17ff147 service placement] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:46.782506 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311e7bd-2d52-4754-9141-9166cdb67c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:46.783503 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311e7bd-2d52-4754-9141-9166cdb67c35 admin admin] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:46.783795 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311e7bd-2d52-4754-9141-9166cdb67c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:46.784127 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311e7bd-2d52-4754-9141-9166cdb67c35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:46.784965 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311e7bd-2d52-4754-9141-9166cdb67c35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:46.793360 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311e7bd-2d52-4754-9141-9166cdb67c35 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:46.870851 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1480/2930] 10.222.0.39 () {60 vars in 1152 bytes} [Tue Jun 7 09:16:46 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:16:46.872113 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a762c-72b2-4545-a2b1-279bae5b4443 None None] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:46.913123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a762c-72b2-4545-a2b1-279bae5b4443 None None] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:47.011426 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011757ed-b478-42de-8b8e-9014c5af6fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:47.012139 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011757ed-b478-42de-8b8e-9014c5af6fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:47.012453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011757ed-b478-42de-8b8e-9014c5af6fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:47.012738 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011757ed-b478-42de-8b8e-9014c5af6fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:47.013656 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011757ed-b478-42de-8b8e-9014c5af6fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:47.015938 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011757ed-b478-42de-8b8e-9014c5af6fe9 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:47.044205 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1451/2931] 10.222.0.39 () {60 vars in 1143 bytes} [Tue Jun 7 09:16:46 2022] POST /identity/v3/users => generated 296 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 09:16:47.048092 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311e7bd-2d52-4754-9141-9166cdb67c35 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:47.056068 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:16:47.064633 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311e7bd-2d52-4754-9141-9166cdb67c35 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:47.077803 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed30be-55c4-435a-9c87-2d41056a9d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:47.079510 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed30be-55c4-435a-9c87-2d41056a9d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:47.079893 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed30be-55c4-435a-9c87-2d41056a9d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:47.080422 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed30be-55c4-435a-9c87-2d41056a9d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:47.081282 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed30be-55c4-435a-9c87-2d41056a9d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:47.083072 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed30be-55c4-435a-9c87-2d41056a9d2a admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:47.101239 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011757ed-b478-42de-8b8e-9014c5af6fe9 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:47.104577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1481/2932] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 09:16:47.119315 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011757ed-b478-42de-8b8e-9014c5af6fe9 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:47.125698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992674d-6ea1-42a4-b0e8-4852ad1b58f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:47.127260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992674d-6ea1-42a4-b0e8-4852ad1b58f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:47.127260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992674d-6ea1-42a4-b0e8-4852ad1b58f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:47.127260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992674d-6ea1-42a4-b0e8-4852ad1b58f2 admin admin] PATH_INFO: `/v3/projects/64ba82e4acfe4d828424adee7175c0e3/users/8c4389df5da24295af8ba25bffd3706a/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:47.144146 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:16:47.152211 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992674d-6ea1-42a4-b0e8-4852ad1b58f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ba82e4acfe4d828424adee7175c0e3, user_id=8c4389df5da24295af8ba25bffd3706a, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:47.155562 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992674d-6ea1-42a4-b0e8-4852ad1b58f2 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:47.173310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1452/2933] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:16:47 2022] PUT /identity/v3/projects/64ba82e4acfe4d828424adee7175c0e3/users/8c4389df5da24295af8ba25bffd3706a/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 09:16:47.177183 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed30be-55c4-435a-9c87-2d41056a9d2a admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:47.191239 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed30be-55c4-435a-9c87-2d41056a9d2a admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:47.313368 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da4be5-a16e-4809-87be-eca4c730e3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:47.314566 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da4be5-a16e-4809-87be-eca4c730e3ad admin admin] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:47.315201 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da4be5-a16e-4809-87be-eca4c730e3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:47.315682 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da4be5-a16e-4809-87be-eca4c730e3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:47.316778 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da4be5-a16e-4809-87be-eca4c730e3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:47.319505 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da4be5-a16e-4809-87be-eca4c730e3ad admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:47.338340 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992674d-6ea1-42a4-b0e8-4852ad1b58f2 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:47.344307 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1482/2934] 10.222.0.39 () {58 vars in 1121 bytes} [Tue Jun 7 09:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 09:16:47.351479 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992674d-6ea1-42a4-b0e8-4852ad1b58f2 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:47.358423 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc2075-a66c-4c2d-9786-e0b53d2bbe6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:47.360801 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2075-a66c-4c2d-9786-e0b53d2bbe6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:47.361336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2075-a66c-4c2d-9786-e0b53d2bbe6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:47.361855 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2075-a66c-4c2d-9786-e0b53d2bbe6f admin admin] PATH_INFO: `/v3/projects/64ba82e4acfe4d828424adee7175c0e3/users/8c4389df5da24295af8ba25bffd3706a/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:47.369440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc2075-a66c-4c2d-9786-e0b53d2bbe6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ba82e4acfe4d828424adee7175c0e3, user_id=8c4389df5da24295af8ba25bffd3706a, role_id=2c1878bf520c4f5f9c3d9a9d2c6e253f)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:47.376161 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc2075-a66c-4c2d-9786-e0b53d2bbe6f admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:47.398208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1453/2935] 10.222.0.39 () {60 vars in 1482 bytes} [Tue Jun 7 09:16:47 2022] PUT /identity/v3/projects/64ba82e4acfe4d828424adee7175c0e3/users/8c4389df5da24295af8ba25bffd3706a/roles/2c1878bf520c4f5f9c3d9a9d2c6e253f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:16:47.820883 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da4be5-a16e-4809-87be-eca4c730e3ad admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:47.822760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-f7da4be5-a16e-4809-87be-eca4c730e3ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:47.837181 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da4be5-a16e-4809-87be-eca4c730e3ad admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:47.915970 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf0bf5-3016-411b-bed3-a0a2428437c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:47.917219 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf0bf5-3016-411b-bed3-a0a2428437c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:47.917599 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf0bf5-3016-411b-bed3-a0a2428437c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:47.918169 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf0bf5-3016-411b-bed3-a0a2428437c2 admin admin] PATH_INFO: `/v3/users/a5bc04ce186c40b0abfe4aa217cb1815` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:47.919701 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf0bf5-3016-411b-bed3-a0a2428437c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5bc04ce186c40b0abfe4aa217cb1815)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:47.922005 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf0bf5-3016-411b-bed3-a0a2428437c2 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:47.989468 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-6baf0bf5-3016-411b-bed3-a0a2428437c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5bc04ce186c40b0abfe4aa217cb1815'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:47.990775 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-6baf0bf5-3016-411b-bed3-a0a2428437c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5bc04ce186c40b0abfe4aa217cb1815'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:47.998931 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-6baf0bf5-3016-411b-bed3-a0a2428437c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5bc04ce186c40b0abfe4aa217cb1815'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:48.012500 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-6baf0bf5-3016-411b-bed3-a0a2428437c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5bc04ce186c40b0abfe4aa217cb1815'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:48.013168 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-6baf0bf5-3016-411b-bed3-a0a2428437c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5bc04ce186c40b0abfe4aa217cb1815'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:48.027600 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1483/2936] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:16:47 2022] DELETE /identity/v3/users/a5bc04ce186c40b0abfe4aa217cb1815 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:16:48.389615 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc2075-a66c-4c2d-9786-e0b53d2bbe6f admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:48.393288 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-9abc2075-a66c-4c2d-9786-e0b53d2bbe6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:48.411593 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc2075-a66c-4c2d-9786-e0b53d2bbe6f admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:48.492317 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:48.493375 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:48.493661 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:48.493943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] PATH_INFO: `/v3/projects/057cc9da8dc64caa89bf47f65fdd24b1` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:48.495405 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=057cc9da8dc64caa89bf47f65fdd24b1)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:48.498185 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:48.579331 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '057cc9da8dc64caa89bf47f65fdd24b1', 'request_id': 'req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:48.592335 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '057cc9da8dc64caa89bf47f65fdd24b1', 'request_id': 'req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:48.609623 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '057cc9da8dc64caa89bf47f65fdd24b1', 'request_id': 'req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:48.629284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] The token cache is being invalidate because project 057cc9da8dc64caa89bf47f65fdd24b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:16:48.629958 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:48.631892 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:48.636243 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1454/2937] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:16:48 2022] DELETE /identity/v3/projects/057cc9da8dc64caa89bf47f65fdd24b1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:16:48.639985 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf0bf5-3016-411b-bed3-a0a2428437c2 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:48.651320 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf0bf5-3016-411b-bed3-a0a2428437c2 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:48.655723 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:16:48.753690 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ce11c-0e9a-48b6-a8cd-fd8c3019f317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:48.754880 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ce11c-0e9a-48b6-a8cd-fd8c3019f317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:48.755886 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ce11c-0e9a-48b6-a8cd-fd8c3019f317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:48.756377 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ce11c-0e9a-48b6-a8cd-fd8c3019f317 admin admin] PATH_INFO: `/v3/users/45532fa2cf1a40aebfae048b7943a42c` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:48.758136 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ce11c-0e9a-48b6-a8cd-fd8c3019f317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45532fa2cf1a40aebfae048b7943a42c)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:48.760274 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ce11c-0e9a-48b6-a8cd-fd8c3019f317 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:48.843496 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0a7ce11c-0e9a-48b6-a8cd-fd8c3019f317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45532fa2cf1a40aebfae048b7943a42c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:48.845949 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0a7ce11c-0e9a-48b6-a8cd-fd8c3019f317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45532fa2cf1a40aebfae048b7943a42c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:48.862527 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0a7ce11c-0e9a-48b6-a8cd-fd8c3019f317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45532fa2cf1a40aebfae048b7943a42c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:48.914267 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0a7ce11c-0e9a-48b6-a8cd-fd8c3019f317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45532fa2cf1a40aebfae048b7943a42c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:48.915934 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-0a7ce11c-0e9a-48b6-a8cd-fd8c3019f317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45532fa2cf1a40aebfae048b7943a42c'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:48.954312 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1484/2938] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:16:48 2022] DELETE /identity/v3/users/45532fa2cf1a40aebfae048b7943a42c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:16:49.324803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:49.326869 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:49.356477 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb727dd-90c1-425d-9b8d-cc7c88b7e2a7 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:49.453698 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:49.454667 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:49.460196 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:49.460490 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] PATH_INFO: `/v3/projects/3dfb813e45574e3e927d6f6e27713220` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:49.462126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dfb813e45574e3e927d6f6e27713220)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:49.464168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:49.475452 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:16:49.538527 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dfb813e45574e3e927d6f6e27713220', 'request_id': 'req-dfe6e02c-03ab-4b5f-873a-03789df2c412'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:49.555786 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dfb813e45574e3e927d6f6e27713220', 'request_id': 'req-dfe6e02c-03ab-4b5f-873a-03789df2c412'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:49.571153 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dfb813e45574e3e927d6f6e27713220', 'request_id': 'req-dfe6e02c-03ab-4b5f-873a-03789df2c412'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:49.593513 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] The token cache is being invalidate because project 3dfb813e45574e3e927d6f6e27713220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:16:49.593909 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:49.595373 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:16:49.599295 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1455/2939] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:16:49 2022] DELETE /identity/v3/projects/3dfb813e45574e3e927d6f6e27713220 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:16:52.070464 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced65ae-31cb-4a08-9ed8-6fb6d6ee9204 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:52.070982 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced65ae-31cb-4a08-9ed8-6fb6d6ee9204 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:52.071211 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced65ae-31cb-4a08-9ed8-6fb6d6ee9204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:52.102694 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-cced65ae-31cb-4a08-9ed8-6fb6d6ee9204 None None] MFA Rules not processed for user `8c4389df5da24295af8ba25bffd3706a`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:16:52.104572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-cced65ae-31cb-4a08-9ed8-6fb6d6ee9204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:52.179577 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1485/2940] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:16:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:16:52.196564 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:16:52.198382 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:52.275882 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6e02c-03ab-4b5f-873a-03789df2c412 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:16:52.281412 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a6165-3fe2-4d54-aa2a-1707a7b4c07d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:16:52.282060 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a6165-3fe2-4d54-aa2a-1707a7b4c07d service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:16:52.282340 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a6165-3fe2-4d54-aa2a-1707a7b4c07d service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:16:52.282611 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a6165-3fe2-4d54-aa2a-1707a7b4c07d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:16:52.284386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-043a6165-3fe2-4d54-aa2a-1707a7b4c07d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:16:52.344891 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a6165-3fe2-4d54-aa2a-1707a7b4c07d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:16:52.346532 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a6165-3fe2-4d54-aa2a-1707a7b4c07d service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:16:52.357524 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1456/2941] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:16:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:17:02.176760 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:17:02.352788 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:17:08.654539 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18008ef9-d4e4-47a4-a828-6d94daf395e1 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:17:08.655071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18008ef9-d4e4-47a4-a828-6d94daf395e1 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:17:08.655071 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18008ef9-d4e4-47a4-a828-6d94daf395e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:17:08.672721 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-18008ef9-d4e4-47a4-a828-6d94daf395e1 None None] MFA Rules not processed for user `97dfa9023e15415d85b6f4650804d6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:17:08.674137 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-18008ef9-d4e4-47a4-a828-6d94daf395e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:17:08.681765 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1486/2942] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:17:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:17:08.992102 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a6165-3fe2-4d54-aa2a-1707a7b4c07d service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:17:08.993541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-043a6165-3fe2-4d54-aa2a-1707a7b4c07d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:17:09.049809 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a6165-3fe2-4d54-aa2a-1707a7b4c07d service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:17:09.053433 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86f4e5-fab6-4dac-a08e-dcbf93641e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:17:09.054014 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f4e5-fab6-4dac-a08e-dcbf93641e95 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:17:09.054295 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f4e5-fab6-4dac-a08e-dcbf93641e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:17:09.054568 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f4e5-fab6-4dac-a08e-dcbf93641e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:17:09.056298 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-eb86f4e5-fab6-4dac-a08e-dcbf93641e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:17:09.066531 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86f4e5-fab6-4dac-a08e-dcbf93641e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:17:09.067925 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86f4e5-fab6-4dac-a08e-dcbf93641e95 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:17:09.076207 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1457/2943] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:17:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:17:09.514546 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18008ef9-d4e4-47a4-a828-6d94daf395e1 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:17:09.515938 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-18008ef9-d4e4-47a4-a828-6d94daf395e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:17:09.586621 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18008ef9-d4e4-47a4-a828-6d94daf395e1 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:17:09.591702 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8524fe-a50c-4514-87d0-916a68fc563e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:17:09.592342 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8524fe-a50c-4514-87d0-916a68fc563e service placement] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:17:09.592626 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8524fe-a50c-4514-87d0-916a68fc563e service placement] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:17:09.592907 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8524fe-a50c-4514-87d0-916a68fc563e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:17:09.594708 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6b8524fe-a50c-4514-87d0-916a68fc563e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:17:09.607275 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8524fe-a50c-4514-87d0-916a68fc563e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:17:09.608854 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8524fe-a50c-4514-87d0-916a68fc563e service placement] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:17:09.618657 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1487/2944] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:17:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:17:19.072803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=26 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:17:19.614445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=35 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:17:23.018493 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86f4e5-fab6-4dac-a08e-dcbf93641e95 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:17:23.033223 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86f4e5-fab6-4dac-a08e-dcbf93641e95 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:17:23.038714 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930d5dd-826a-4683-aad5-5112c08868ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:17:23.039365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930d5dd-826a-4683-aad5-5112c08868ab service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:17:23.039645 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930d5dd-826a-4683-aad5-5112c08868ab service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:17:23.039917 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930d5dd-826a-4683-aad5-5112c08868ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:17:23.041893 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0930d5dd-826a-4683-aad5-5112c08868ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:17:23.054840 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930d5dd-826a-4683-aad5-5112c08868ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:17:23.056386 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930d5dd-826a-4683-aad5-5112c08868ab service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:17:23.068123 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1458/2945] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:17:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:17:26.111986 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8524fe-a50c-4514-87d0-916a68fc563e service placement] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:17:26.121688 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8524fe-a50c-4514-87d0-916a68fc563e service placement] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:17:26.125230 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7becf-857f-4fa3-8b6e-b2c6fdcead88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:17:26.125909 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7becf-857f-4fa3-8b6e-b2c6fdcead88 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:17:26.126238 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7becf-857f-4fa3-8b6e-b2c6fdcead88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:17:26.126582 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7becf-857f-4fa3-8b6e-b2c6fdcead88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:17:26.136700 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7becf-857f-4fa3-8b6e-b2c6fdcead88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:17:26.138156 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7becf-857f-4fa3-8b6e-b2c6fdcead88 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:17:26.147406 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1488/2946] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:17:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:17:33.063260 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:17:36.143304 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:17:51.278162 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930d5dd-826a-4683-aad5-5112c08868ab service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:17:51.287556 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930d5dd-826a-4683-aad5-5112c08868ab service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:17:51.291956 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd0e9e-4983-4014-afc4-933f4c733b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:17:51.292588 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd0e9e-4983-4014-afc4-933f4c733b4b service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:17:51.292930 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd0e9e-4983-4014-afc4-933f4c733b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:17:51.293284 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd0e9e-4983-4014-afc4-933f4c733b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:17:51.295041 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-53fd0e9e-4983-4014-afc4-933f4c733b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:17:51.304440 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd0e9e-4983-4014-afc4-933f4c733b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:17:51.305899 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd0e9e-4983-4014-afc4-933f4c733b4b service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:17:51.313967 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1459/2947] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:17:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:18:01.310427 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=6 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:19:25.457515 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd1470-8921-4796-97f5-8906c14482f1 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:19:25.458035 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd1470-8921-4796-97f5-8906c14482f1 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:19:25.458215 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd1470-8921-4796-97f5-8906c14482f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:19:25.476038 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-b7fd1470-8921-4796-97f5-8906c14482f1 None None] MFA Rules not processed for user `97dfa9023e15415d85b6f4650804d6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:19:25.477922 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b7fd1470-8921-4796-97f5-8906c14482f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:19:25.485314 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1489/2948] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:19:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:19:35.470415 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=3 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:20:03.530000 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd0e9e-4983-4014-afc4-933f4c733b4b service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:20:03.558783 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd0e9e-4983-4014-afc4-933f4c733b4b service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:20:03.568771 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1483d5-c649-423e-a04c-88ce02b3d646 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2a2bf913d1412f996acf4401ba7673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:20:03.569459 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1483d5-c649-423e-a04c-88ce02b3d646 service nova] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:20:03.569745 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1483d5-c649-423e-a04c-88ce02b3d646 service nova] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:20:03.570052 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1483d5-c649-423e-a04c-88ce02b3d646 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:20:03.574168 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5b1483d5-c649-423e-a04c-88ce02b3d646 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:20:03.587302 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1483d5-c649-423e-a04c-88ce02b3d646 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:20:03.588872 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1483d5-c649-423e-a04c-88ce02b3d646 service nova] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:20:03.600217 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1460/2949] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:20:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:20:13.595073 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=12 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:21:19.411561 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e29e01-a99e-48e4-805b-fb3dd5412b13 None None] REQUEST_METHOD: `POST` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:19.412076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e29e01-a99e-48e4-805b-fb3dd5412b13 None None] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:19.412076 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e29e01-a99e-48e4-805b-fb3dd5412b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:19.441613 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.auth.core [None req-e5e29e01-a99e-48e4-805b-fb3dd5412b13 None None] MFA Rules not processed for user `97dfa9023e15415d85b6f4650804d6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80229) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 09:21:19.442790 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e5e29e01-a99e-48e4-805b-fb3dd5412b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:19.452900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1490/2950] 10.222.0.39 () {58 vars in 957 bytes} [Tue Jun 7 09:21:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 09:21:19.492494 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1483d5-c649-423e-a04c-88ce02b3d646 service nova] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:19.493943 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-5b1483d5-c649-423e-a04c-88ce02b3d646 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:19.593615 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1483d5-c649-423e-a04c-88ce02b3d646 service nova] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:19.597160 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89bf99-3e7b-4ae4-8b2e-65282e7c5c2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6943045d594d729fc7c4b15cbaa07e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:19.598047 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89bf99-3e7b-4ae4-8b2e-65282e7c5c2d service cinder] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:19.598336 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89bf99-3e7b-4ae4-8b2e-65282e7c5c2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:19.598611 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89bf99-3e7b-4ae4-8b2e-65282e7c5c2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:19.600942 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-0f89bf99-3e7b-4ae4-8b2e-65282e7c5c2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:19.626565 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89bf99-3e7b-4ae4-8b2e-65282e7c5c2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:19.627974 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89bf99-3e7b-4ae4-8b2e-65282e7c5c2d service cinder] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:19.638753 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1461/2951] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:21:29.450784 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=9 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:21:29.633985 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=49 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:21:52.551711 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e29e01-a99e-48e4-805b-fb3dd5412b13 None None] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:21:52.554001 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-e5e29e01-a99e-48e4-805b-fb3dd5412b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:52.598323 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e29e01-a99e-48e4-805b-fb3dd5412b13 None None] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:21:52.603797 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7b975-1de2-456f-badb-8848642bf156 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:21:52.604445 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7b975-1de2-456f-badb-8848642bf156 service placement] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:21:52.604737 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7b975-1de2-456f-badb-8848642bf156 service placement] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:21:52.605012 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7b975-1de2-456f-badb-8848642bf156 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:21:52.606814 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-a9d7b975-1de2-456f-badb-8848642bf156 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:21:52.619055 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7b975-1de2-456f-badb-8848642bf156 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:21:52.620625 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7b975-1de2-456f-badb-8848642bf156 service placement] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:21:52.630838 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1491/2952] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:21:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:22:02.626186 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=18 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:22:53.247370 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89bf99-3e7b-4ae4-8b2e-65282e7c5c2d service cinder] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:22:53.266237 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89bf99-3e7b-4ae4-8b2e-65282e7c5c2d service cinder] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:22:53.270347 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f7949-b3d4-4a9a-a473-c9d2ff38eb96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:22:53.270988 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f7949-b3d4-4a9a-a473-c9d2ff38eb96 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:22:53.271273 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f7949-b3d4-4a9a-a473-c9d2ff38eb96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:22:53.271546 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f7949-b3d4-4a9a-a473-c9d2ff38eb96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:22:53.278823 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f7949-b3d4-4a9a-a473-c9d2ff38eb96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:22:53.280241 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f7949-b3d4-4a9a-a473-c9d2ff38eb96 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:22:53.288867 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1462/2953] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:22:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:23:03.284915 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=10 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:23:06.804536 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7b975-1de2-456f-badb-8848642bf156 service placement] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:23:06.813916 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7b975-1de2-456f-badb-8848642bf156 service placement] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:23:06.817462 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a9a99-b123-417c-9b12-982f6b275bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:23:06.818120 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a9a99-b123-417c-9b12-982f6b275bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:23:06.818408 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a9a99-b123-417c-9b12-982f6b275bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:23:06.818687 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a9a99-b123-417c-9b12-982f6b275bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:23:06.826875 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a9a99-b123-417c-9b12-982f6b275bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:23:06.828301 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a9a99-b123-417c-9b12-982f6b275bc1 service neutron] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:23:06.837069 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1492/2954] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:23:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:23:16.833258 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=6 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:23:20.100126 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f7949-b3d4-4a9a-a473-c9d2ff38eb96 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:23:20.108830 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f7949-b3d4-4a9a-a473-c9d2ff38eb96 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:23:20.113027 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a317e0-c0a4-46d3-b4b1-53f1bd2fd631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ddf89deb28749eda2d26ad66bc282f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:23:20.113712 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a317e0-c0a4-46d3-b4b1-53f1bd2fd631 service neutron] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:23:20.113974 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a317e0-c0a4-46d3-b4b1-53f1bd2fd631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:23:20.114255 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a317e0-c0a4-46d3-b4b1-53f1bd2fd631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:23:20.116393 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-69a317e0-c0a4-46d3-b4b1-53f1bd2fd631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:23:20.201199 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a317e0-c0a4-46d3-b4b1-53f1bd2fd631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:23:20.202541 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a317e0-c0a4-46d3-b4b1-53f1bd2fd631 service neutron] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:23:20.222266 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1463/2955] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:23:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:23:23.095416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a9a99-b123-417c-9b12-982f6b275bc1 service neutron] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:23:23.108944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a9a99-b123-417c-9b12-982f6b275bc1 service neutron] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:23:23.118590 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50997ba-9ac1-4fd5-b516-117469f4dd73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:23:23.119534 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50997ba-9ac1-4fd5-b516-117469f4dd73 service placement] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:23:23.119956 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50997ba-9ac1-4fd5-b516-117469f4dd73 service placement] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:23:23.120352 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50997ba-9ac1-4fd5-b516-117469f4dd73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:23:23.122545 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c50997ba-9ac1-4fd5-b516-117469f4dd73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:23:23.138085 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50997ba-9ac1-4fd5-b516-117469f4dd73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:23:23.140640 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50997ba-9ac1-4fd5-b516-117469f4dd73 service placement] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:23:23.157572 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1493/2956] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:23:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:23:26.253228 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a317e0-c0a4-46d3-b4b1-53f1bd2fd631 service neutron] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:23:26.270025 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a317e0-c0a4-46d3-b4b1-53f1bd2fd631 service neutron] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:23:26.281638 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8336e83-350a-46b8-9356-ec2cc4f65373 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:23:26.282419 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8336e83-350a-46b8-9356-ec2cc4f65373 service placement] REQUEST_METHOD: `GET` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:23:26.282803 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8336e83-350a-46b8-9356-ec2cc4f65373 service placement] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:23:26.283310 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8336e83-350a-46b8-9356-ec2cc4f65373 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:23:26.285976 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-b8336e83-350a-46b8-9356-ec2cc4f65373 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:23:26.303488 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8336e83-350a-46b8-9356-ec2cc4f65373 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:23:26.306301 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8336e83-350a-46b8-9356-ec2cc4f65373 service placement] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:23:26.313842 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50997ba-9ac1-4fd5-b516-117469f4dd73 service placement] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:23:26.316902 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-c50997ba-9ac1-4fd5-b516-117469f4dd73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:23:26.336610 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50997ba-9ac1-4fd5-b516-117469f4dd73 service placement] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:23:26.337365 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1464/2957] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:23:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:23:26.343644 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:23:26.344627 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:23:26.344978 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:23:26.345381 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] PATH_INFO: `/v3/users/97dfa9023e15415d85b6f4650804d6d9` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:23:26.347003 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97dfa9023e15415d85b6f4650804d6d9)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:23:26.349449 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:23:26.404680 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97dfa9023e15415d85b6f4650804d6d9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.407793 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97dfa9023e15415d85b6f4650804d6d9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.415047 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97dfa9023e15415d85b6f4650804d6d9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.428084 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97dfa9023e15415d85b6f4650804d6d9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.428847 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97dfa9023e15415d85b6f4650804d6d9'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.441919 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1494/2958] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:23:26 2022] DELETE /identity/v3/users/97dfa9023e15415d85b6f4650804d6d9 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:23:26.618437 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8336e83-350a-46b8-9356-ec2cc4f65373 service placement] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:23:26.629822 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8336e83-350a-46b8-9356-ec2cc4f65373 service placement] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:23:26.692549 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:23:26.693453 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:23:26.693741 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:23:26.694021 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] PATH_INFO: `/v3/projects/de4176cee52a461fa7eeb6a471e82593` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:23:26.695208 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de4176cee52a461fa7eeb6a471e82593)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:23:26.697127 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:23:26.744278 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de4176cee52a461fa7eeb6a471e82593', 'request_id': 'req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.748104 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:23:26.750538 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de4176cee52a461fa7eeb6a471e82593', 'request_id': 'req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.759070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de4176cee52a461fa7eeb6a471e82593', 'request_id': 'req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.770279 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] The token cache is being invalidate because project de4176cee52a461fa7eeb6a471e82593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:23:26.770569 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.771209 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.772759 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1465/2959] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:23:26 2022] DELETE /identity/v3/projects/de4176cee52a461fa7eeb6a471e82593 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:23:26.776896 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:23:26.778324 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:23:26.790900 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801a797-e4cd-482e-b8b5-05ab2061ba48 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:23:26.847425 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:23:26.848214 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:23:26.848751 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:23:26.849029 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] PATH_INFO: `/v3/users/8c4389df5da24295af8ba25bffd3706a` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:23:26.850236 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c4389df5da24295af8ba25bffd3706a)` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:23:26.852172 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:23:26.896041 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c4389df5da24295af8ba25bffd3706a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.896710 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c4389df5da24295af8ba25bffd3706a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.904253 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c4389df5da24295af8ba25bffd3706a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.909243 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:23:26.920159 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c4389df5da24295af8ba25bffd3706a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.920840 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.notifications [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c4389df5da24295af8ba25bffd3706a'} {{(pid=80229) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:26.932987 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1495/2960] 10.222.0.39 () {58 vars in 1223 bytes} [Tue Jun 7 09:23:26 2022] DELETE /identity/v3/users/8c4389df5da24295af8ba25bffd3706a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:23:27.106325 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] Authenticating user token {{(pid=80228) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:23:27.107807 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.fernet_utils [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80228) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:23:27.120895 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac88afa-d121-4ec6-8e9e-ee5c5932df69 admin admin] Validating token access rules against request {{(pid=80228) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:23:27.179080 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50aa837-c0de-4ebe-846a-401fb8300fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70de4148c8f4696ac913ae91f5a81cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ddc110e64f52436a913830733ab91055', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80228) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:23:27.179939 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50aa837-c0de-4ebe-846a-401fb8300fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:23:27.180224 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50aa837-c0de-4ebe-846a-401fb8300fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:23:27.180847 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50aa837-c0de-4ebe-846a-401fb8300fc0 admin admin] PATH_INFO: `/v3/projects/64ba82e4acfe4d828424adee7175c0e3` {{(pid=80228) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:23:27.182070 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50aa837-c0de-4ebe-846a-401fb8300fc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64ba82e4acfe4d828424adee7175c0e3)` {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:23:27.183953 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50aa837-c0de-4ebe-846a-401fb8300fc0 admin admin] RBAC: Authorization granted {{(pid=80228) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:23:27.229885 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c50aa837-c0de-4ebe-846a-401fb8300fc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64ba82e4acfe4d828424adee7175c0e3', 'request_id': 'req-c50aa837-c0de-4ebe-846a-401fb8300fc0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:27.235445 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c50aa837-c0de-4ebe-846a-401fb8300fc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64ba82e4acfe4d828424adee7175c0e3', 'request_id': 'req-c50aa837-c0de-4ebe-846a-401fb8300fc0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:27.242716 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c50aa837-c0de-4ebe-846a-401fb8300fc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64ba82e4acfe4d828424adee7175c0e3', 'request_id': 'req-c50aa837-c0de-4ebe-846a-401fb8300fc0'} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:27.263397 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c50aa837-c0de-4ebe-846a-401fb8300fc0 admin admin] The token cache is being invalidate because project 64ba82e4acfe4d828424adee7175c0e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80228) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 09:23:27.264078 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c50aa837-c0de-4ebe-846a-401fb8300fc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:27.264401 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG keystone.notifications [None req-c50aa837-c0de-4ebe-846a-401fb8300fc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80228) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 09:23:27.266896 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: [pid: 80228|app: 0|req: 1466/2961] 10.222.0.39 () {58 vars in 1232 bytes} [Tue Jun 7 09:23:27 2022] DELETE /identity/v3/projects/64ba82e4acfe4d828424adee7175c0e3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 09:23:36.929443 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:23:37.250744 dv-cinder-733062-30-iscsi devstack@keystone.service[80228]: DEBUG dbcounter [-] [80228] Writing DB stats keystone:DELETE=9,keystone:SELECT=56,keystone:INSERT=2 {{(pid=80228) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:26:58.487946 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] Authenticating user token {{(pid=80229) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:26:58.489416 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:58.559339 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d695a-a278-41c0-92c8-db29c61c6ef6 admin admin] Validating token access rules against request {{(pid=80229) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:26:58.619222 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eebee0-396d-4ba2-b0c5-3920e1e299c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a7d1e8a0f4b4511a7dc8b27651744c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b861f95bf3498d990785ba72474d82', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80229) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:26:58.619964 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eebee0-396d-4ba2-b0c5-3920e1e299c0 service placement] REQUEST_METHOD: `GET` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:26:58.620358 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eebee0-396d-4ba2-b0c5-3920e1e299c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:26:58.620783 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eebee0-396d-4ba2-b0c5-3920e1e299c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80229) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:26:58.622944 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.fernet_utils [None req-b5eebee0-396d-4ba2-b0c5-3920e1e299c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80229) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:26:58.634541 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eebee0-396d-4ba2-b0c5-3920e1e299c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:26:58.636125 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eebee0-396d-4ba2-b0c5-3920e1e299c0 service placement] RBAC: Authorization granted {{(pid=80229) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:26:58.645983 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: [pid: 80229|app: 0|req: 1496/2962] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jun 7 09:26:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:27:08.641743 dv-cinder-733062-30-iscsi devstack@keystone.service[80229]: DEBUG dbcounter [-] [80229] Writing DB stats keystone:SELECT=58 {{(pid=80229) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}